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I. GENT.IAL irrTRODUCTIOj; 

PROJECr CAREZR/CAN (C-rccr Analysir, Network, OEG-73-3967) 
was funded lor a twelve r;onth period ccmriOncing in July, 1973 and 
terrr.inat iny Juno 30, 1974. The aoals of the project were focused 
on the dcvolop::;cnt of inf orrrat ir-n and delivery systems which would 
cut ricrc'iiis crade lines and provide career infornation to students 
and tcachcr::j in qradcs K-l-i . The structure of this Project was 
intended to r-noblc further development of the behavioral objective 
product to include four colur-^ru:; cf curricular inf orii\-^t ion and to 
abror::- the- onccinc; pil-^t activities: that had been started under 
PROJFCI" CARi:bR- 

PROJF.cr CAREER (Co-puter Airsxsted Research lor Educational 

Relevance) war. funded for a three year period corrr.cncinc July 1, 1971, 

and term.inatina June 30, 107^1. Tne tjrinary ir.i^sion of this project 

v;as "to Gevelo::^ test, and evaluate a process dc;sicjned to facilitate, 

at the. secondary and post-secondary school levels, the learning of 

marketable knowledcie, sk.ills, una cittitudes wb.ich are needed to secure 

2 

eniployrnent in selected current and er.ercjiM'j occupot ions . During 
the first fiscal year of the Project, two additional grant ; roposals 
were v;ritten and funded. PROJECT CAREER/GU1DA::CE {OEG-0-72-4G5 1) 
was funded from July 1, 1972 to N'over.ber 30, 1973, with the termination 
date for final reporting purposes extended to June 30, 1974. PROJECT 
CAREERALANDICAPPED (OEG-0~72-5170) was funded from July 1, 1972, to 
June 30, 1974, with an e>rt:ension for seme final internal rejort 
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writing through September, 1974. 

The above-mentioned ongoing pilot activity was to be implemented 

by PROJECT CAREER/GUIDANCE and PROJECT CAREER/IIANDICAPPED . CAREER/ 

GUIDANCE had as its mission to take the knov/ledge bcse (a behavioral 

objective product) and capabilities of the parent project (now 

identified as PROJECT CAREER/DEVEL0PME::.T ) and ''develop, test and 

evaluate a new approach to ^ roviding students from levels K through 

post-sccondary with bettor knowledge about the spectrum of occupational 

opportunities that really c>:ist, information about the educational 

requirements which relate to them, and objective data on their own 

abilities and linitations in relation to the requirements of occu- 

3 

pat ions of interest." 

CAREERAlAirniCAPPED also war? to draw on the data base developed 
by PROjECa^ CAREER/DEV:LOr:-iEOT and had as itc. goal •'meeting the 
individual needs of pcr.*Tons with physical, emotional and other 
disabilities, so that thoy can, insofar as they are able, have 

educational and occupational opf»ortunit ies equal to those of students 

4 

who are not handicapped." V/hcn the Third Party Evaluator began 
the evaluaticM in January 1973, theso three components (CAREER/ 

DEVJ:LOP:-u:rrr, carei:;r/guid;.::ce, carkhr/hamdicapped) conc^tituted the 

totality of PROJECT ChRETAi and each component was dependent on 
the product dovclopir.cnt for the acco.Tplishmcntr of its goals in 
the testing or piloting phar.c. 
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Because each coir.ponent of PROJECT CAREER as well as PROJECT 
CAREER/CAN were in. some way dependent on the core project, the 
separately reported projects need to be set in this overall 
context. Some overlapping in these reports is inevitable, but rhe 
evDiuators have made an effort to focus the final reports only on 
those aspects of the con.ponent being treated in each report. 

This repurt is a sum/native evaluation of PROJECT CARELk/CAN, 
covering its one year funding period. The report will describe arid 
evaluate the development of the four column process which completes 
the behavioral objective data base. The pilot testing of this data 
in some high school classrooms wi^'l also be evaluated on the 
following pages . 




II. ABSTRACT 

PROJECT CAREER/CAN was funded for the period of July 1, 1973 
to November, 1974. The major goal of. this Project was to complete 
the development of four column curricular information for each of 
the PROJECT CAREER behavioral objectives, and to complete the pilot 
testing of those 30' s in secondary classrooms. These objectives 
were absorbed from another component of PROJECT CAREER and were 
utilj*;od by this Project after they had already been implemionted 
for at least 12 monrhs. 
PEPJ^OmvIEL 

Personnel for implementing this Project were drawn from the 
central staff of PROJECT CAREER, The key person ^vas the Curriculum 
Administrator, Mr. Roger Ritch, who was rer:;ponsible for the four 
coluir.n development work. To asi^ist him in this work^ he had two 
interns and appro:<imat ely 40 i^ub-ccntr acted personnel trained and 
supervised in the writing of the four column material. la connection 
with the secondary school pilot testing, there were eight classroom 
teachers involved in two high schools. 
PROJECT 

IMPLEME^jTATI O^I 

Four colum.n development v;as well organiiicd with a recruitment 
and training system that enabled the writing of the four colum.ns to 
kee[> pace with the supply of throe part BO's comdng from earlier 
stages of the product developm.ent . Several manuals were developed 



to aid in the writing of one of the four colunms. Ot>er manuals 
for the other four columns were brought to the draft or planning 
stage. 

The pilot test, although producing some positive response 
from the teachers involved, was not an adequate test of the final 
product complete with four columns. Only two occupational groups 
were tested and most of the EO's u-^ed for tcjiting wero incomplete 
or inadequately packaged for use. This Project did not put any 
additional resources into piloting although piloting was part of the 
original objectives of the Project. 
F TILINGS 

1. The additional four columns of information added to the 
three part Mager objective represents a significant and 
innovative advance for occupational education. This is 
a highly tran^portriblo product . 

2. Full development of the process for producing the four 
columns is not yet complete. The models tor develop- 
ment of additional manuals and standards of quality 
control are ready and v.ill be developed. 

3. A data bank of completed four column objectives for 116 
occupations is complete. 

4. A sample package of the product indicates that there is 
still much variability of quality in the four columns. 
The weakest columns were prerequisite skills and component 



tasks. The concepts column was the most fully developed. 

5. Computer retrieval capability is now available for 
several useful configurations of data . Additional column 
coding is necessary to make the computer capability 
fully useful. 

6. The limited pilot testing eistablished that the data, at 
this time, har> high impact on occupational instructors 
and moderate im.pact on academic teachers . The pilot 
test did not produce measurable impact on students at 
this tLme. 

REcor-r.EN T) AT io::s 

In view of the above findings, the Third Party Evaluator 
recommends the following: 

1. That the Project not release any more data for testing 
until any batch to be released has been subjected to 
additicnal editing^ ap.d any gaps in information have 
been filled. 

2. That the Project give high priority to completing the 
final remaining development cteps for the four columns, 

namely the completion of the remaining manuals and 
coding . 



3. That upon the com.pletion of the remaining four column 
manuals, the completed four colur.n objectives be re- 
edited and coding added where missing . 




4. That additional pilot testing be backed up with careful 
in-service training for the personnel conducting the 
test, that regular follow-up contact be maintained to 
monitor progress, and that adequate supporc personnel 
be made available to consult on instructional design 
development using the Project data. 

J. That additional programs and configurations of data 

retrieval be limited until the initial retrieval programs 
can be tested for their feasibility and potential user 
demand . 

6. That the Project p.a'kc a major investment in developing 
a few alternative ways of scquenciny and packaging 
the data to maximize the value and usefulness to 
potential users. 

C ONCLUSIO N 

PROJECT CAREER/CAN was funded July 1, 1973, and in June, 1974, 
war? cvr luatcd on its three major objectives which were to complete 
the addition of four columns of information to the PROJECT CAREER 
behavioral objective, to com.putcrize these data for effective user 
retrieval, ana to pilot te^'t the uc- of the product in the classroom. 
All of these objectives were inherited from ongoing components of 
PROJECT CAREER that began twelve to eighteen months before this 
Project was funded. The funding from this Project was entirely 
directed toward the first two objectives, with the third objective 




allowed to continue on its own froip previous funding. The evaluative 
findings substantiate that the first two objectives were effectively 
accomplished and the product at this point is about 90% complete. 
A sufficient data bank has been established to warrant further 
testing. The pilot test established the feasibility of using this 
product in the classroom, but was too limited and ineffectively 
monitored to provide more than rudimentary data for further product 
development • 
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Ill, HISTORICAL HIGHLIGHTS 



PROJECT CAREER/CAN (i*V361049L, Grant #)EG-'72-2967) was originally 
written in proposal forn about January 1973, and was then eventually 
funded under the aegis of the new PROJECT CAREER Director, Mr* 
Lano, to begin July 1, 1973, and to end November 30, 1974. Formal 
Third Party Evaluation covered a twelve month period through 
June 30, 1974. 

PROJECT CAREER/CAN was designed to allow for an extension of 

certain aspects of the work already developed under PROJECT CAREER/ 

DE\nBLOPMEOT and PROJECT CAREER/GUIDANCE. At the outset of this 

Project in July 1973, each of the CAREER/CAN objectives had already 

been initiated.: The mission of this Project, then, was to carry 

through with the ir.plenentat ior. of objectives that were part of 

earlier funded proposals. 

PROJECT 
OBJECT I\"E5 

This Project had three basic objectives: 

• To develop the basic three part PROJECT CAREER behavioral 
objectives, validated for occupations, into an educationally 
relevant objective with four additional colums of 

curr icular inf i:;rr::at ion . 

• To expand the computer capabilities for providing user 
information based on these objectives. 

e To pilot test the capabilities c-f using these objectives 
in curriculum: development, instruction, and guidance. 




The baseline for these developments was to be the availability 
and/or the continuing development of validated three part objectives 
from PROJECT CAREER/DEVELOPFJINT • Pilot testing and computerization 
of the data had already been implemented by PROJECT CAREER for more 
than twelve months prior to the funding of this proposal- 

During the ensuing twelve months of this Project, significant 
strides "were made in the developm;ent of the four colum.ns for the 
bo's. The pilot activities, hovevejL, were not augmented and LEA 
personnel continued on their own with developnents they had worked 
on during the four week workshop of Jul\, 1973. This w^orkshop was 
a major event designed to further the piloting objectifies of CAREER/ 
CAN. 
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IV, EVALUATION DESIGN 

The evaluation of PROJECT CAREER essentially involved the^ 
development of systematic information gathering procedures that would 
be useful for ongoing formative evaluation of a research and develop- 
ment project • The Project existed to develop a product and to tost 
the feasibility of its use in the school setting. Although there 
was an early ir.plication in the Project's aims that a final possible 
outcome could take the form of student learning, in reality, the 
Project never did cone close to attaining this as an ultimate 
outcome. Thus, the outcome or summative evaluation problem was 
primarily limited to a study of the final product and the process 
through which it was develcpcd. Fart of the evaluation of that 
product came from: data supplied by potential users of the product as 
they tested it out in its early form in the classroom.. 

Since the Prcj<:ct never had a systemiatic, carefully monitored 
field testing component wit> specific targeted outcomes, the most 
system.atic evaluation could come fromt m.onitcrinc the data developm.ent 
process • vrhile this process v:as itself subject to experim.entat ion and 
change, there was at least a systematic, flew charted procedure 
whereby the product was to be developed and produced. Until the 
final phases of the Project, therefore, evaluation concentrated on 
process and was form.ati\^e in nature, "J he evaluation design was 
intended to provide the Project with ongoing evaluative feedback 




ERIC 



that could be used to improve and enhance the development of the 



product and its testing • 



A comprehensive written evaluation design was developed 



based largely on process and product output objectives, which 
provided a very systematic basis or information gathering. The 
nature of these objectives and the Project itself lent itself 
toward much qualitative evaluatic... (Quantitative outputs were 
in the form, of cumulative counts. In their Jiesicn, the evaluators 
strove for the most systematic collection of information possible 
in a form, that would be m.ost useful for ongoing decision making 
regarding the Project's aims. Some of this information gathering 
took the form of ongoing consultation w^ith m.em.bers of the Project 
Staff • Eventually, the Evaluators developed a form of interim, 
reporting in which full reports ar.d shorter miem.os were prepared on 
specific asp-ects of the Project in order that this information would 
reach the Project soon enough to be of value. 

In response to the form.ative recomjnendat ions that were m.ade 
in NEESI's quarterly and interim: reports, the opportunity for a 
great deal of dialogue develop-ed between the Project Staff and 
ICEESI Consultants. This process, in turn, provided the source 
of follow up activities on the part of the evaluators as needed 
changes and modifications v;ere identified. Information was continually 
sought from, the Project as to whether or not changes had been m.ade 
as veil as for documentary evidence that changes had taken place. 
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INFORMATION 

GATHERING 

TECHNIQUES 

The following is a listing of various information gathering 
techniqxies employed by the ©valuators in i.^iplcnent ing the general 
evaluation plan described above* 

• Interviews with key Project personnel. 

• Interviews with LEA personnel. 

• Interviews with sub-contracted pert:.onnel {e.g. con- 
verters, validators , coders , etc. ; 

• Structured surveys or quest ionnaixcc of certain Project 
personnel. 

• Inspection of documents, both internal, iTianacenent 
docurr.ents and products snd activities produced by 
the field testing. 

• Consultation with outside experts. 

• Product evaluation fcllcving standards established 
by instructional product design experts. 

• On-site observation of field testing activities, 
workshops, conferences, and personnel training sessions. 

aRITTEN 

D0CU!1E:CT at I ON 

To pro\'ide for crnsistcnt and systematic feedback of this 
inforr^ticn to the Project, the Evaluators submitted the following 
written docur.enrs: 

• Five written evaluation design proposals* 

• Seven quarterly reports. 

• Inter ir. fcrr.ative evaluation reports on the following 
aspects of PROJECT CAREER: a) two reports on validaticn? 
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b) two reports on four column development; c) one 
report on general product development and packaging; 
d) one report on pilot activities; e) one product 
evaluation reports 

• Whenever key Project central staff were interviewed, 

memos summarizing the interviews and the understandings 
reached were sent to PROJECT CAREER within one week. 

Some of the complexity of monitoring all the Project's 

available sources of information can be appreciated by the fact 

that over the life of the Project, there were nearly 400 different 

persons involved in some aspect of the Project. Besides the basic 

full time Project staff based in the Randolph, MA. headquarters, 

there vras a large group of affiliated LEA personnel in over ten 

u?>fferent schools, as well as hundreds of sub-contracted personnel 

that perfcrrried the t asks of writing objectives, validating objectives, 

coding objectives, and writing four coluir.ns for the objectives. 

The evaluation team made an effort to have some contact, if only 

through a survey form, with all of these persons. 
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V> PROGRAM OBJECTIVES 



AS was pointed out in the introduction to this report, PROJECT 
CAREER/CAN was funded for the purpose of augmenting, in three 
main areas, the work of PROJECT CAREER: 

• Four colunn developnent of BO's. 

• Expansion of computer capabilities for information 
retrieval 

• A feasibility test of the four column data for use 
in K~12 classroom and career guidance programs. 

In the following section of this report, we shall specify 

each of the three Project objectives and give sumjrative comm^ents 

on the status of those objectives after twelve months of Project 

activity. More detailed data will be provided in subsequent 

sections of this report, 

STATUS OF 
OBJECT Pt'ES 



1. GrVEN 5 0,000 VALIDATED BEHAVIORAL OBJECm-ES, DE\TLO? i:^ RELATION 
TO EACH, THE BASE DATA FOR INCLUSICi; U^IH THE CATEGORIES CF (A^ 
?RF --REQUISITE LEAPwCINSS , (3) C0y.:'0l7EZn TA^K, {Cl EITl'IRO^^MIZ^^T AND 
{D) CONCEPTS. 

The figure of 50,000 behavioral objectives was a targeted 
figure established early in the life of PROJECT CAPJEER. This 
figure, however, was never rorc than an estir.ate of what the Project 
might be able to develop. By the tir.e PROJECT CAREER/CAN was 
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actually fur.ded, whe parent proje 
four colur.n cbiectives. 
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ct. had produced about 55 0 co-cleted 
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By June 30, 1974, the total number of four column objectives 
completed vas about 9500, covering all but a few of the 118 occu- 
pations on the PROJECT CAREER list. Other significant develop- - 
nents were the completion of coded manuals for related concepts from 
the fields of mathematics, science, and business; the completion of 
a coded manual for prerequisite skills in r;athemat ics ; the initial 
draft of a manual for coding alternative environments; and the 
development of a model for completing additional coded manuals to 
include the com. pone nt tasks colum.n. 

The recruitment of four colurn v;riters as well as their 
training of the writers was developed to the point where production 
could be kept up to the availability of validated three p.^rt 
objectives in any of the PROJECT CAREER occupations. 
2. GI\'E^: 5 0,000 VALIDATED EEiL?iVICRAL OBJECTI\:nBS WITH RELATED 
I^CFCRMATICN (A,3,C,D, H: =1), DE\^0? AZ^ AUG>E!rrED COMPUTER CAPA- 
BILITY FOR STORAGE AIvD RETRIPJAL OF INFORMATION BY ELEMENT ARY% 
JUICIOR HIGH, SECO^SARY A!0 POST SECONDARY COUNSELORS AND TETiCHERS . 

Com.puterizinn the data as it was developed had already been 
build ir.to the PROJECT CAREER system. The com:puter hardware vas 
aucm.ented before this present Project vras funded to allov for a 
disc system with added storage capacity. Nearly IC^z of the 
ccm.puter needs of the Project curing the tvelve m.onths of this 
funding period vere taken up v:it:h data develcpment. Er>:cept for one 
or two isolated rnsrances, no actual data retrieval for classroom. 
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or guidance use was undertaken during the life of the Project. 

However, as we shall document in a separate section, the 
needs of users for certain combinations of data were prograrrjmed and 
the planning of computer time and capacity for meeting user needs 
in further testing of the product was accomplished. 

3. PROVIDE IN-SEKVICE TRAINING TO ELEIENTARY, JUNIOR HIGH, SECONDARY 
AND POST SECOND'ARY COUNSELORS AND TEACHERS IN THE PILOT DISTRICTS 
IN THE irriLIZATION OF THE COMPUTERIZED RELATED INFORMATION DATA 
DE\'ZLOPED IN OBJECTI\'E #1. 

By mid-way through the Project year, it was evident to both 
the evaluators and the Project Staff that no further work after the 
SuTJT.er v:orkshop had actually been carried out with the pilot LEA's. 
At the -rcing of the evaluators:, the Project Directors revised the 
objectives for the pilot testing to focus on the role of the pilot 
testing as it irpacted on data development. Final summative evalu* 
ation of pilot actiLvities v/as therefore based on the revised objec- 
tive, and the details of this evaluation v:ill be provided in a 
subsequent section of the reports 

The oc;;- G-ct::Lvc as i":. vas \s'rittcn above, hovever, vas never 
ir.ple-entcd. No ccrpleted four colur.n objectives vith complete 
cc.T.putcr retrieval capabilities vere ever developed earlv enouch to 
pro'/ide an adequate test of the product as its potential use. 
Thus, all piloting vas e>rtrenely liniited in the e:<tent to which 
it could ser\'e as a *;:est for the fully developed -odei orioinallv 
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envisaged in the language of the proposal. 

Another factor that accounts for the limited attainment of 
this objective, was the fact that the final budget for this Project 
did not actually fund this objective. The only budgeted costs 
associated with this objective were a portion of administrative 
costs that went inro lijr;iited planning and supervision of the LEA 
effort • 

If we were to limit our evaluative considerations to the 
actual relationship between the stated objectives, the budgeting and 
disbursement of funds assigned to this Project, and the actual 
observed level of Project staff involvement in the pilot LEA"s, 
then we would conclude that this objective of CAREER/CAN was 
actually never iriplerr.ented. In point of fact, however, there was 
pilot activity going on sustained by funds from other components 
of PROJECT CAREER, and on v:hich we shall report in subsequent 
sections • 
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VI. ADMINISTRATION AMD PERSONNEL 

There were two categories of personnel involved with PROJTCT 

CAREER/CAN (excluding LEA personnel) . One was the full tin\e staff 

at Project headquarters in Randolph. The other category were 

coinpriscd of part time sub-contracted personnel that wrote the 

four columns. This section will discuss these categories 

separately, as well as the general administration of the Project. 

Because the same central staff performed functions associated 

with all components of PROJECT CAREER, the description and evalu-- 

at ion of administration and central staff will be similar for each 

component of the Project. 

OVERALL 
ADMI^TISTRATION 

The adm.inistrative structure of PROJECT CAREER is represented 
in. part by the organizational chart on the fclloving pace (Figure 1) . 
This chart indicates job titles that are associated with various 
functions required to carry cut the overall operations of the Project 
Nearly all of the full tir.e professional staff were assigned to 
duties represer.ted by at least tvo job titles, vrritten. -^ob descrirj- 
tions were developed for each of these positions. 

The job titles and the vrritten specifications adequatelv 
i^'^^proccnt Lhe range of functions needed by the Project to accom.ulish 
most ox a.ts oojectj^ves. !rEESI's Third Party Evaluators have some 
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expertise in sophisticated research functions, and in curriculum 
and instructional design technology. But these are matters of 
judgment and hindsight based on the performance of the already 
hired staff. 

In the same vein^ there was not always a close match between 
the training and experience specified for the job title and the 
training and experience of the person hired for, or appointed to 
the job task. Again, we must allow for som.e flexibility to account 
for the market in available personnel, and the fact that m^any of the 
capabilities needed for success in any of the Project tasks had to 
be developed through experience. In the opinion of the Third Party 
Evaluators, the Project staff were generally competent, enthusiastic, 
and dedicated. They were able to implement m.cst of the Project 
objectives to a successf'^I conclusion. Eiit the Project staff did 
not, in the final analysis, possess all of the capabilities which 
cor Id have been used. The two major weaknesses v:ere noted above. 
FISCAL 



account ar.r was hired who installed a fiscal 
mar.agement system that enabled the careful control and accounting of 
the funds allocated to the Project. The final decision to hire the 
accountant was due to the Project Administrator's realization that 
this was a most important step to take as soon as he became the 
overall adm.inisrrator in !^ovember, 1973. This system, included a 
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bi-weekly report that provided the director with the following 
inf ormat ion: 

1. Budgeted dollars for the total funding period by line 
Item andpercentagc of the total. 

2. Dollars spent by line item during the two week report 
period. ^ 

3. Percent of the total line item budget for that period. 

Percent of the total spent during that period by -.n- 
item. J- . A. - 



4. 



5. Cumulated dollars spent to date. 

6. Percentage of total line item budget spent to date. 

The balance remaining of the budgeted dollars for each 
line item- 
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line item. 

The system provided for the use of a triplicate voucher 

that enabled the careful tracking of all money disbursed by the 

coded budget numbers. These vouchers were the basis for the fiscal 

records. The vouchers were the basis for the fiscal records. The 

vouchers could be compared with actual amounts paid out by the checks 

drawn against the Project's account. The Third Party Evaluators 

have received all of these bi-weekly reports. 

STRONG POSITIVE 
ADMINISTRATION 

Effective project management includes a well developed 
structure for allocating personnel to tasks, including recruiting 
and training, and a sound fiscal management system. m addition 
to these tangible structures, there are the intangible aspects of 

27 
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establishing a good working environment^ providing for effective 
interpersonal cOifimunicat ions , establishing decision making procedures 
that are geared to the nature of the Project and the professional 
staff, supervising staff and functions so that quality is maintained 
and projects are completed, and in general, prcvidiiig the Project 
with sound leadership and good public relations. 

The information summarized above indicates that the Project 
did develop reasonable, successful administrative and fiscal 
structures. In general, competent personnel were hired and assigned 
to tasks that they wure capable of performing. Quality control was 
maintained to some degree at ^ oast as evidenced by the departure 
from the Project, by agreement, of any staff person who was not 

performing adequately in an important task. 

REPORTING AND 
RECORDS 

With the exception of fiscal reporting and personnel records, 
which were described under Administration, the major reports and 
records maintained by the Project were the following: 

1. Quarterly and Annual reports. 

2. Data production repori-s, periodically prepared for 
internal circulation. 

3. Logs of data development processes such as validation, 
coding, four column writing. 

4. News and Views , a newsletter circulated to all LEA 
personnel and others affiliated with and interested in 
PROJECT CAREER. 



The major record keeping problems were discussed under 
ADMINISTRATION AND PERSONNEL. With regard to the above list, the 
judgment of the Third Party Evaluators is that these reports were 
relevant to and adequate for the needs of the Project in these 
areas . 
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VII. GENERAL FINDINGS 

The summative details of our evaluative findings will be - 

orovided in this section and will be keyed to each of the three 

major Project objectives - four cnlu.T.n dc vt:lopment, computer 

capability, and pilot testing. 

FOUR COLWW 
DEVELOPMENT 

The complete PROJECT CAREER behavioral objective, as it was 
eventually defined after several developmental stages, consists of 
a task outline listing the major responsibilities and duties of the 
occupation; three part Magcr objectives based on the duties or 
duty examples; codes indicating the attainability of the objective 
by different handicapped populations; and four columns of additional 
information that enable the objective to be related to curriculum 
and instruction. The handicapped coding vas in independent opera- 
tion that did not alter the objective in any way. But, the addition 
of the four columns was a significant advance enabling the teaching 
of occupational skill objectives or job task statements directly in 
the classroom. 

The Third Party Evaluators made two extensive analyses of 
the basic four column model for the purpose of formative evaluation. 
These were submitted as separate reports on December 27, 1973, and 
March 15, 1974. The model was cxamir.od, along with a sample of 
completed data, by a leading expert in instructional design. Dr. 
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Leslie Briggs of Florida State University, In considering this 
model, its potential value for curriculum development and instruc- 
tional design, and other possible alternatives for designing such 
objectives, THE EVAMJATORS, WITH 'rHE SUPPOPT OF DR. BRIGGS, CONCLUDED 
THAT THIS BEHAVIORAL OBJECTIVE PRODUCI WITH THE FOUR COLUMNS OF 



INFORMATION IS AN OUTSTANDING INNOVATION. 



The four columns are defined as follows, taken from the 
four column writers training manual (see Appendix, page A~23): 

• Pro-Requisite Lcn r nings - Those skills, knowledges, 
understandings and attitudes, which will not be 
taught during the treatment, but which are necessary 
for a student to possess in order to facilitate the 
acquisition of the new behavior. 

• Component Tasks - The individual activities which are 
necessary for the completion of the listed behavior 
when that behavior must have component tasks per- 
formed in sequence. 

• Environment - The alternative environments which may be 
utilized for acquis^ition of the listed behavior, i.e. - 
media, mode of instruction. 

• Concepts - The abstract rules, principles, laws, and/or 
generalizations which are related to the performance 

of the listed behavior, Thene are listed for purposes 
of providing for the transfer of learning as well as 
for interdisciplinary approach to instruction. 

• Production Strategy - The detailed objectives from the 
original proposal called for the four columns to be 
written by teams of writers combining occupational 
expertise and educational expertise. As the writing 
was fully implemented, however, these teams, with a 
few exceptions, were never formed and all four columns 
were written by the same person. The writers were 
recruited from occupational educators or persons with 
both job and teaching experience in a given instruc- 
tional area represented by the BO's assigned to the 
writer . 
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Ultimately, there will be manuals developed for each column 
with coded entries bo rhat a writer can generally select the neces- 
sary information from the manual, and enter it along with the code 
on the grid sheets. These manuals are not intended to be limiting, 
however, and when a manual docs not contain information that a 
writer believes should appear in a given column, that information 
is to be included. 

For the current production year, the only fully developed 
manuals available were those for the concepts column. Here the 
writers had access to lengthy lists of coded concepts for mathe- 
matics, science, and business. An inspection of these manuals 
revealed them to be well organized and extensive. The concepts 
were alphal -tically by categories. 

The first of several planned manuals for the pre-requisite 
skills colurrin was completed for muth skills, /additional nanuals 
are planned for science skills;, coirTiunicat ion skills, and occu- 
pational skills. 

An alternative cnvironrr.cnts m.anuai has been completed in 
draft form, but will not be coded or published until the information 
can be organized by USOE Occupational Clusters. This organization 
will enhance the value of the manual, since potential users can 
consider a number of alternative learning environments (which 
includes tools and equipment) for the different occupational 
clusters. No work has been done, as yet, on a manual for the 
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corr.ponent casks colunn. Such a manual requires a simple to complex 
sequencing operation of the available data before component tasks 
can be placed in a manual. This eventual sequencing will be a 
major step forward for the product, for it will enable the relating 
of data to learning hierarchies. 

Four Column Writers 
Written job specifications for four column writers incliade 
the following: 

1. Current industrial-occupational experience in the area 
assigned for curriculum development. 

2. Instructional or job training experience. 

3. The ability to work as part of an interdisciplinary 
team charged with m^aking joint decisions. 

4. A knowledge of minimal related area requircn.ents for 
successfully performing job skills. 

5. A knowledge of developing curriculum materials for 
instruct ional use. 

6. A willingness to expend a "reasonable" amount of time 
in the development of four column data. 

7. A willingness to base a workinij relationship with PROJECT 
CARKKR on a successful performance level. 

Mr. Roger Ritch, the Project's Curriculum Administrator, 

developed a detailed recruiting and training plan. Writers were 

successfully recruited from newspaper advertisements and through 

extensive contacts with the regional vocational-technical schools. 

A one page personal data sheet was collected for each person. 
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Applicants were interviewed and, simultaneously given individual 
training. Quality control, and retention as a writer was based on 
successful continuing performance based on a 100% edit of the 
returned material. 

Since the writing was never implemented on a team basis, 
qualification #3 was never required. The evaluators inspected the 
complete records of all active and inactive four column writers, 
and found that there were no exceptions to the qualifications required 
for items #1 and #2. For exeimple, a writer for the instructional 
area of inhalation therapist was listed as a respiratory therapist 
with 7 years experience; a writer for auto and diesel mechanics 
was an automotive teacher witli 1!3 years experience; a writer for 
nursing, nursing assistant, and medical laboratory technician was 
a registered nurse with 14 yearn experience, including the teaching 
of licensed practical nurses. 

As of June 1974, there were 31 active four column writers and 
13 who had written but were currently inactive. Their years of 
experience ranged from tv.o to more than twenty years with a median 
of about seven years. As of the end of June 1974, the instructional 
areas covered by these w^riters included about 80 of the 116 occu- 
pations . 

Product ivity 

Product j.vity ov3i . six mouths of the Project averaged 

about 300 EC's per week ur*c . c'^i-rc were sufficient validated three 
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part BO*s available. During one high production period, as many as 
500 a week were completed. By the end of the Project, the system 
was capable of recruiting and training a sufficient range of man- 
power to keep up with the production pace of validated three part 
objectives. Total production of completed four column BO's by June 
30, 1974, was approximately 9500. Mr. Roger Ritch estimated that, 
by September, BO's for all 116 occupations would be complete with 
four columns. Estimates of that final figure ranged from 12,000 
to 16,000. 

The Product 

No careful, objective evaluation of the quality and utility 
of this product is possible at this point. Some information on 
utility will be discussed under findings from the pilot testing. 
This testing was limited to just two occupational groups, elec- 
tronics and business. 

There are a few comments that can be made about the quality 
issue. First, the quality of the final product depends, to some 
extent, perhaps a large e:<tent, on the quality of the three part 
BO. A check of a final batch of completed BO's for automobile 
mechanics, supplied by the Project for purposes of evaluating a 
sample of the final product, indicated that there is still wide 
variation in the quality of the three part BO. 



Feedback from the four column editors, based on their editing 
of the columns and their training and follow-^up contacts with the 




four column writers, indicated that the order of difficulty in 
writing the columns follows the order of the columns: prc-requis ite 
skills is most difficult to write, component tasks next, then 
cnvironm.ent , and concepts is the easiest. The latter is also the 
only column with completed manuals for the writer * 

Our scanning of the sample of auto mechanics BO's would 
tend to confirm this order of difficulty. Pre-requisite skill 
colixnns were highly variable, many containing items lacking the 
stems "use of," **knowlcdgc of," or "ability to," as prescribed 
in the editor's manual. A typical item found in many of the auto 
mechanic prc-rcquisitc skills columns v;as "Sth grade reading." 
Presumably this relates to the reading level of the instructional 
manual for car repair. However, at least one user of these BO's with 
the educable mentally retarded, suggested that this was a meaningless 
pre-^requisite since many of the skills could still be learned 
without this reading ability. Sometimes a concept v;ould appear by 
itself in the pre-requirite skill column and again in the concept 
column . 

Component task columns covered r vyide range of detail. 
Some simply repeated the task statement. Some specified that the 
automobile instruction manual be followed. And some were detailed 
task breakdowns of the more complex task statement in the three 
part BO . 
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Obviously, the only effective evaluation of the quality of 
this four colunin information can come from the occupational educ^tcr 
who will use the data for instructional purposes. As we shall detail 
below, testing of this data has been extremely limited to date. The 
evaluators have reason to believe that the saniple package of data 
we were shown does not represent the best that is available from the 
Project at this time. 

Finally, we note again that the process is still not yet 

fully developed. In the absence of the remaining manuals for use 

by writers, the information produced will not be as systematic as 

it needs to bo, nor will variability in quality be reduced to a 

more tolerable level. However, it is clear from an inspection of 

the final product that the omount, extent, and general quality of 

information now available for teaching these skills is far in 

excess of what most teachers presently have available, and would 

undoubtedly be welcoitic support in any occupational teacher's 

instructional planning. 

COMPUTER 
RETRIEVAL 

The second of the three major objectives of CAREER/CAN 
carries the following specifications from the original proposal: 

1. Retrieval of pre-requisite learnings for a single 
behavior or clusters of behaviors . 

2. Retrieval of concepts for a single behavior or a 
cluster of behaviors. 
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3, Search across behavioral objectives and/or occupations 
common to any single or group of pre-requisite learnings 
or concepts • 

4, A commonalities retrieval which will search out behaviors 
common to a wide range of occupations. 

These specific objectives actually cover only a small number 
of the many useful and important ways in which information, based 
on these data, could be retrieved from the computer. As the Project 
stands at the conclusion of twelve months, the lack of coded pre- 
requisite column material (with the exception of math skills) makes 
it impossible to implement the above objectives which pre-suppose 
such codir^vj. Such coded information is available for the concepts 
CvJium.n only. One pilot retrieval effort has already been imple- 
mented for a user in a pilot school. As the other two columns are 
coded, this will e:<pand the possibilities even more. 

The commonalities retrieval is fully operational at this 
time, although no commonality searches have been m.ade to determine 
vliat the data might look like. From our study of the process and 
the product as it has been developed, the evaluators believe that 
the most significant commonality information for instructional 
purposes will eventually come from the data in the four columns 
and not from the occupational (job task) skills contained in the 
three part objectives. But such conmonality searches must await 
the full development of coded information for each of the four 
columns. Except for the concepts column, this represents several 
norc months of development. 
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four column writers, indicated that the order of difficulty in 
writing the columns follows the order of the columns: prc-requis ite 
skills is most difficult to write, component tasks next, then 
cnvironir.ent , and concepts is the easiest. The latter is also the 
only column with completed manuals for the writer* 

Our scanning of the sample of auto mechanics BO's would 
tend to confirm this order of difficulty. Pre-requisite skill 
colxmns were highly variable, many containing items lacking the 
stems "use of," "knowledge of," or "ability to," as prescribed 
in the editor's manual. A typical item found in many of the auto 
mechanic pre-requisitc skills columns v;as "8th grade reading." 
Presumably this relates to the reading level of the instructional 
manual for car repair. However, at least one user of these BO*s with 
the educable mentally retarded, suggested that this was a meaningless 
pre-requisite since many of the skills could still be learned 
without this reading ability. Sometimes a concept v;ould appear by 
itself in the pre-requinite skill column and again in the concept 
column . 

Component task columns covered r vyide range of detail. 
Some simply repeated the task statement. Some specified that the 
automobile instruction manual be followed. And some were detailed 
task breakdowns of the more complex task statement in thn three 
part BO . 
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Obviously, the only effective evaluation of the quality of 
this four column information can come from the occupational educr^twr 
who will use the data for instructional purposes. As we shall detail 
below, testing of this data has been extremely limited to date. The 
evaluators have reason to believe that the saniple package of data 
we were shown does not represent the best that is available from the 
Project at this time. 

Finally, we note again that the process is still not yet 

fully developed. In the absence of the remaining manuals for use 

by writers, the information produced will not be as systematic as 

it needs to bo, nor will variability in quality be reduced to a 

more tolerable level. However, it is clear from an inspection of 

the final product that the omount, extent, and general quality of 

information now available for teaching these skills is far in 

excess of what most teachers presently have available, and would 

undoubtedly be welcome support in any occupational teacher's 

instructional planning . 

COMPUTER 
RETRIEVAL 

The second of the three major objectives of CAREER/CAN 
carries the following specifications from the original proposal: 

1. Retrieval of pre-requisite learnings for a single 
behavior or clusters of behaviors. 

2. Retrieval of concepts for a single behavior or a 
cluster of behaviors. 




3, Search across behavioral objectives and/or occupations 
common to any single or group of pre-requisite learnings 
or concepts . 

4, A commonalities retrieval which will search out behaviors 
common to a wide range of occupations. 

These specific objectives actually cover only a small number 
of the many useful and important ways in which information, based 
on these data, could be retrieved from the computer. As the Project 
stands at the conclusion of twelve months, the lack of coded pre- 
requisite column material (with the e;<ception of math skills) makes 
it impossible to implement the above objectives which pre-suppose 
such codir^vj. Such coded information is available for the concepts 
CvJium.n only. One pilot retrieval effort has already been imple- 
mented for a user in a pilot school. As the other two columns are 
coded, this will expand the possibilities even more. 

The commonalities retrieval is fully operational at this 
time, although no commonality searches have been m.ade to determine 
vliat the data might look like. From our study of the process and 
the product as it has been developed, the evaluators believe that 
the most significant commonality information for instructional 
purposes will eventually come from the data in the four columns 
and not from the occupational (job task) skills contained in the 
three part objectives. But such connonality searches must await 
the full developiT:cnt of coded information for each of the four 
columns. Except for the concepts column, this represents several 
more months of development. 
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COMPUTER 
FACILITIES 

This section of the report, fron page 34 to 44 , reviews the 
present status of PROJECT CAREER and PROJECT CAREER/CAN data processing 
activities together with planned future activities. The report was 
written by Mr. Larry English, the computer specialist hired by NEESI. 
Comments arc provided on the feasibility of proposed activities as 
well as the progress made to date in the data processing area. The 
report provides a brief summary of information presented in previous 
reports on this subject, dated June 14, 1973 and December 28, 1973, 
as a context for evaluating the present status of work- 

Thc report is based on information provided in interviews 
with Mr. John Post, Peter Cook and William Shaw, conducted during 
July, 1974 and on documents provided by Peter Cook. 

The observations and rccom.:iendat ions contained in this 
section arc listed in full on pages 42-44 and the text provides 
background information relative to the observations and recomraen- 
dationc. These observations and recommendations can be suirjr^ar izcd 
as follows: 

• Productivity from the computer operation has greatly 
increased, and the data processing operation now sup- 
ports project requirements fully. 

• Some projected future project activities requi-.. , 
however, development of some related data processing 
f aci lit ies v;hich do not now exist . 
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• Although there are minor gaps as noted above, future 

plans for data dissemination and testing are generally 
soundly based on the presently available computer 
hardware and software. 

PRESENT ST/iTUS OF 
CAREER/CAN DATA 

PROCESS I r:G 

In this section, wc will review the present status of ^r.u^^tfer 
operations for PROJECT CAREER and PROJECT CAPJEER/CAN and prescut 
some cor.Tncn^-3 about the rvolutioa of the computer operation. 

Present Activit y 

PROJECT' CA?Kr^. and PROJECT CAREER/CAN arc concerned with 
establishing cata base of approximately 20,000 behavioral objec- 
tives v.hich have been "validated" in the sense that their relevance 
to one or mere of a group of occupations has been established. 
Validation of objectives originally involved a several tier screening 
proce::r: . The validation procccs was an extensive and cumbersome 
one thaL required corjputcr support to make it even partially 
nirnrKjC able . 

Initially, behavioral objectives were first written by persons 
in various cciucational aroaci. The original "yield" in tern;s of 
valiaatcd ob3CCtives froni the rav; data was relatively lov;. In 
addition, the project, in its initial stage, was plagued by large 
backlo'js in the data processing activity arxd seening comir.unicat ion 
difficulties between data processing personnel and other project 
staff racmbcrs . 
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In the initial data generation system, an objective was 
first keypunched, and a computer file of such objectives was prepared. 
Objectives were processed in approximately 2000 objective "batches." 
The objectives in thca.^ files were sorted by instructional area 
subgroups according to the U.S. Office of Education instructional 
coding system. "ID" numbers were also assigned to the objectives. 

After this initial computer processing, reviewers for each 
of fifteen occupational clusters determined whether the objectives 
were relevant to their clusters. A computer file of the ID numbers 
of objectives assigned to each cluster was then produced. This file 
was then p-sscd again.^t the initial "batch" of objectives, and 
questionnaires were produced for a seco-^d level of screening. 
Objectives which passed through both levels of screening were 
coiisidGr-ad v^alida'tied . 

Once objectives %vere validatca, they were annotated and also 
classified according to rhe ability of handicapped persons to perform 
tho:z. Figure 3 illustrates the initial data generation procedure. 

prcccnt, objectives are initially described bv persons 
working in the various cccupaticns of interest to the Project. This 
has greatly improved data quality, at least in term.s of the "vield" 
in validated objectives. Last year, after two years of work, the 
Project had o~ly 500 validated objectives. Nov:, after three years 
of effort, there are 10, COD validated objectives. 
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The improvement in data quality has riade parallel efforts 
possible in data generation. Now the basic validation process, the 
annotation (four colunn) process, and the classification for handi- 
capped persons can all be done at essentially the same tiiTiC without 
first waiting for cor.plGtion of the validat c prcrcss. 

HARDWARE FACILITIES 
AlTD PR0GR?.VS 

The data generation systcn presently runs on a Kone^^'ell 240- 
computer v;ith a 9S, CCD character rr.errory, four tape drives operating 
at 66, COO character per scccnd transfer rates and a density of 
SOO bits per inch, and 55.2 millicn characters cf on-line disk 
storage cn three reriovablc pack drives. 

The data gcr.cratirn systcn. involves apprcxirr-ately fifteen 
frequently used prograr.s out cf a library of approxir^ately 40 progra 
v:hich arc used frcr. tir.o to tiro and apprcxir.at ely nine additional 
progran.s v;hich v-crc vrittcn for specir.l purposes and are rarely 
us od or h r.v c be c n used o nl v once. 




up to a standard that would support operation of the data generation 
programs by progranvT,cr s and computer operators with no previous 
knowledge of the development of the programs. 

GENERAL 

OBSERVATIO!^ 

The software used for creating the file of validated objectives 
has stabilized and reached a level of quality such that it adequately 
supports project objectives with respect to data processing for 
data generation purposes. File structures and objective coding 
scher.es have been simplified and improved as the project has pro- 
gressed. The administration of the computer activity has also im.proved 
greatly so that coals an.-'c prioritLies for the computer personnel are 
now clearly established. 

A previous tendency to over use the computer for w^ork best 
?^r-crmeu ry ncn-compuwcrizeu printing and reproduction uevices 
has also been eliminated v;ith resulting i^mprcvemcn::s in data processing 
efficiency. 

ACTIVITY 

In this section will describe CAREER and CAP^EVCa:T da^a 
processing plans as they relate to data dissemination, evaluation 
of data effectiveness, and maintenance of the m-aster data file. 
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Data Dissrninat ion 



During the current fiscal year, the project plans to distribute 
data to the approximately 350 school districts in the state • 
Twenty-four of these districts will be selected for special pilot 
evaluation of data ef f ect iveness • 

To do this, the project's data processing group will carry 
out two kinds of activities: 

• Data will be cor.piled in "off the shelf" catalogs 
in multiple formats as by job title, by instruc- 
tional procrar?., by conTon skill eienent, or by handi- 
capped grouping . 

« Lata v;ili also be co-piled in special catalogs so long 
as the requests for corr.pilaticns of this sort can be 
supported by existing progra-T.s or easily developed 
new prog rarr.s . 

So far there has been only iir.iteu user experience with 
project data so that the fcrr.ats and crr.zc-nt for the "off the 
shelf" catalogs have been largely dcLcr.T.ircrl by the project staff. 



It is contcrr.platcd that pre-prir.ted data catalogs could be 




one half will be drawn from populations with normal needs • 

The primary intent icn of the evaluation process is to determine 
whether or not access to PROJECr CAREER data results in curriculum 
improvement. A secondary intention in evaluation will be the 
beginnings of a process to refine and improve the master data file. 

Project personnel contemplate some use of data processing 
techniques in the evaluation process, but no specific plans have 
been made. Such planning will be undertaken after scm.e initial user 
responses have been obtained • 

Maintenance 

Plans for maintaining the project data base are presently 
mccr.plcte. It is assumed that data development will continue, and 
that user reaction ^-ill also permit constant ii^provement of the - 




few as ten percent of the objectives initially defined were actually 
validated, yields o± approximately ninety-eight percent are now 
being experienced • Processing backlogs are largely eliminated, with 
keypunching being, possibly, the only remaining bottleneck in data 
flow. 

2. HARDWARE A:^:D SCFTVrARS FACILITIES ARE E^rTlRELY ADEQUATE FOR 
PRESE!^^ A:aD PROJECTED DATA PROCESS II^G OPERATIONS. 

The present work loaS and the work projected for the next 
fiscal year arc easily supported by the present Konewell configu- 
ration. Existing prograrns and prograrr. docur.cntat ion ore also 
adcqua^G for these operations. 

3. AiTHou:^:: ?la:t::i::3 for data disse.minatic:: is RELATrrzLY comflete, 
DATA PRCcrssiiTG RELATED pla::^:i!:g for rj-iiirrrrrAiccE of the data ai^td 
E^v'ALTATic:: CF USER e:«:perie::ce FvEqtifjes ATTE:n?io::. 



As nctcd on par. e , data processing support is assuned to 




such requests for both technical (prograrrjming) feasibility and 
intrinsic worth. This approach will limit the workload for special 
compilations to a manageable level while still insuring that realizable 
and worthwhile requests for such compilations are honored. The 
concentration of data distribution in a single center is also wise^ 
since it permits, in this initial pilot phase of usage, proper 
control over the quality of the data base and prevents any possible 
"drifting apart" of the content of multiple copies of the data base. 
This later prchlen v;ould be a serious one if the project master 
file vvcr:^ tD be maintained s im.ult aneouc iy at several com.puter 

■*.a'— ^wiCo • 

5. BETTER CCMPLTIZR TROZ^^^l F^XILITIES FOR DATA EDITING SEOULD 

The task of maintaining project d^ta can be m.et for the tim.e 
beinr \:Lzr, the existing lim.itecl facilities for record deletion and 
insertion described in Section n". E\^entually, hovrever, data 
irprcvcment faciliticc m.u£t be developed so that records can be 
individu::i:y corrected in a single program, run and so that ccmm,on 
corrections or additions can be made to groups of records in a 




PILOT 
TESTING 

The original objectives of the pilot test, as written in the 
PROJECT CAREER/''CAN proposal, emphasized activities that were guidr.nce 
oriented and directly related to those objectives already imple nented 
and funded by the CAREER/GUIDANCE corr.ponent of PROJECT CAREER. 
Because the CAREER/CAN budget never funded these -objectives, and 
because they were fully evaluated in the report on the Vroject 
which did fund them, there will be no reference in this report to 
thoire pilot activities. 

/lowever, the pilot use of the four column BO's in secondary 
school classrooms is directly relevant to the major thrust of 
CAREEpyCAN, the full four colum.n developr:enx: of the BO's for 
instructional use. This pilot test was "inherited" from PROJECT 
CAREERA>E\'EIiOPVE:Jr, but is best evaluated ar.d reported on in the 



context of C?^REER/CA^r^ despite 



frcm. this Project. 



lack of direct fundinc suoocrt 



The pilot test relevance to the m,issicn of CAREER/CAIv was 
r.ade explicit in December I5""3, :in a revised set of objectives 
prepared for the second ^ry school component. These new objectives 
identified the m.ajor purpose of the pilot test as a mean^s "to 
determ.ine whether PROJECT CAREER, data is appropriate for the 
imolemiCntat ion in the classroom;." 



The socciLCiC 



oo^ectives arawn up for this pilot test, which 



were then communicated to the Lz^. r>ersonneI involved 
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1. 



Determine if the overall formav of PROJECT CAREER data 
is useful as a basis for curriculum development* 



2. Analyze ways in which occupational teachers use each- 

of the parts of the PROJECT CAREER behavioral objective 
format, i.e. Condition, Performance, E>rtent, Pre- 
requisite Learning, Component Tasks, Alternative 
Environments, and Related Concepts. 

3^ Anal^-ze ways in vtiich interdisciplinary team members 
use Condit ion , Performance , Extent , Pre-requis ite 
Learning, Component Tasks, Alternative Environments, 
Related Concepts. 

4. Identify potential problems in converting raw data into 
'} earning packages . 

5. Develop processes to convert raw data (PROJECT CAREER 
f crmat ) into individualized learning packages . 

6* Test the use of PROJECT CAREER data based lesson plans 

in different kinds of classroom modes, i.e. small group, 
large group, individual instruction, etc. 

7. Identify problems with ir.plenent ing PROJECT CAREER data 
based lessons in the classroom. 

6* Determine teacher reaction to using lessons developed 
from PROJECT CAREER data. 

9* Determine student reaction to working with instructional 
r.aterials based on PROJECT CAREER data • 

The basis for testing the above set of objectives w^s e>rtremelv 

lir.it ed, as will be seen from, details on the pilot given below. The 

Third Party Zvaluators developed a questionnaire (See Appendix, page A-7) 

that W3S designed to elicit as r.uch inforr:ation as possible bearing 

on the attainm.enu of these objectives. However, the information on 

specifics is very spotty for reasons we shall m.ake clear. What is 

unequivocal, however, from, this verv limited =nd ooorlv structured 
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and directed pilot test, was that the PROJECT CAREER data is indeed 
useful to teachers . 

Pilot Schools and Students 

The original PROJECT CAREER proposal specified that pilot 
testing would be conducted in an urban systen, a suburban system, 
and a regional vocat ional-technical school systen. In the Spring 
of 1972, the three LEA"s were selected, with Springfield, Milford, 
and Blue Hills Regional Vocational-Technical School representing 
the three specified settings. 

Frorc the outset of the involver.ent with the LEA*s, there were 
p:oblen:s that affected the data develcpnient w^rk of PROJECT CAREER 
and created poor relationships with the LEA personnel. 3y the time 
CAREER/CAN was funded, the pattern of the pilot tests wa5^ already 
established and che difficulties were part of the total history of 
PROJECT CAREZPv These problems have all been evaluated in r^revious 
Third Party E\^aiuator reports, including one interir. forr.ative 
report on piloting thar was part of the ongoing evaluation of this 
Project . 

As a resulr of these ta-rled relationships, there were only 
two pilot tear.s operative at the Secondary level, one in Springfield 
and one in Milford, by July 19"^3, when C^R'E:E:ii/C?{i: began, and when a 
na^icr s™crraer workshop was held to being developing curriculum: 
iTaatiirials Survey data were collected fro~ a curriculur. coordinator 
and a tcac:her in Randolph High School (a ''feeder^ school for the 
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Blue Hills Region) • No actual pilot use of these data, however, 
was ever made in that school, and the infornation available from 
the surveys does not contribute anj-'thing to the evaluative data - 
collected frop Springfield and Milfcrd. We shall therefore limit' 
our reporting to those two systems. 

Pilot Plan 

The original pilot plan called for an interdisciplinary team 
approach in which the ttan leader or core teacher would be the 
occupational teacher and ^hc other r.erbers of the teair. would be 
fror; acader.ic disciplinea . Ihe occupational teacher would receive 
bo's for his occupational instructional area and the acader.ic 
reachcrs would r*akc their instruction.^1 inputs primarily from the 
pre-rcquisite skills and concepts coluT.ns. 

The plan envisaged an intact group of students studying under 
the occupational teacher who would rove as a group to the academic 
teachers on the team. This plan would enable the academic teachers 
to maximize the relevance of their subject r.atter for these students, 
since teaching would be based on the concept and skill learning 
directly tied to behavioral objectives b-eing used by the occupational 
teacher. The relevance of this learning for career decision making 
would be further enhanced by a plan to relate -the activity to the 
guidance com.ponent through the use of student inventors- files* 



These files woulc const itut: 



occuoational skills {-^cb tasks] learned bv the student 



enr . -^n^s 5ki-l« 
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inventory would be computerized and serve as a basis for searching 



the data bank for all occupations to which that skill development 

would be relevant. This exploration component was to be facilitated 

through the development of Career Information Centers (CIC) . 

This is a very powerful and veil developed model. However, 

a number of missing elements made it LT.possible to test and, from. 

that point of view, insured the failure of the piloting. 

• Complete four column objectives were never available 
at the outset of the pilot so that the teaching team»s 
never had fully developed data with which to work. 

p The plan to have the sam.e students in both the occu- 
pational class and the acGGem.ic classes could net be 
implemented in the pilot schools because of schedule 
difficulties. Therefore, the teamwork anticipated by 
the model could only be im.plemiented in piecemeal fashion 
with only some of the students. No real im.pact of the 
interdisciplincHry approach was possible to assess. 

0 Student inventory file forr^ were actually developed 

but they w^ere never cor.puterized. Because com.plete and 
systematic data were lacking and only im.plem.ented in the 
classroom, on a spotty basis, the student inventory of 
skill development was meaningless and, to our knowledge, 
was never actually carried out by any students. 

c Career Inform.ation Centers were established in the pilot 
schools, but their use v:as never integrated into the 
skill ceveiopment process for the reasons outlined above. 
For the sam.e reasons, there was never any m-eaningful 
interfacing bctvreen the pilot testing teaching com- 
ponent and the pilot testing guidance com-ponent of 
the pilot testirig. 

foat, then, remained of the pilct cffcrt that cculd contribute 

:he oilot 



• Two occupational teachers received PROJECT CAREER data 
for occupations in their instructional areas and these 
data were used in their classrooms. 

• The teams' academic teachers made conscientious and, - 
in some cases, imaginative efforts to utilize PROJECT 
CAREER data, when available, in order to complement the 
instructional efforts of the occupational teacher. 
When data were lacking, the academic teachers developed 
their own materials that were faithful to the concept 
or intent of the data use. 

From this effort, it was possible to extract some minimal, 
but useful, information as to the value of the data to the classroom 
user. Other than second-hand feedback from the teachers, no meaning- 
ful data could be collected relative to the impact on students. 

Springfield Pilot 
A young electronics teacher was the core teacher for the 
pilot project at Springfield Trade High School. His team was 
composed of a mathematics, physics, and English teacher. His 
final project report, detailing his experiences during the 1973-74 
school year, provides excellent insight into how he and his team 
experimented with the use of PROJECT CAREER data. This report is 
included in the Appendix, page A-2 . 

Computer printouts of several hundred electronics BO's were 
made available to the Springfield team. During the summer workshop, 
July 19 73, they worked on the development of some learning activity 
packages (LAP's). These were to be implemented in the Fall. By 
all accounts, the LAP's were a failure, and after the first half 




of the school year, further efforts in that direction were abandoned. 
The teachers reported that the students did not like the LAP"s. 
The math teacher belie^'cd that the data for math^related LAP"s 
was weak, in that it lacked higher mathematics concepts. The data 
war m'^ch better for physics, especially the pre-rcquisite skills 
and concepts columns. 

The evaiuators xnspected the LAP's developed after the summer 
workshop and found them to be quite woak. The teachers reported 
that the promised technical assistance was never forthcoming and 
only the physics teacher was confident of his ability to write 
good lap's. In any case, LAP's were not developed beyond that r : 
the summer workshop. Those that wore developed by the English 
teacher showed good imagination and creative potential, but there 
was no connection in any of these LAP's to PROJECT CAREER data. 

Because of th:^ aforementioned scheduling problems and the 
weaknesses in the available data, the piloting contributions of 
Spr inqf icld ' s academic teachers to the Project was limited. Again, 
we must emphasize that this result was not due to any failure of 
effort on the part of the academic teachers. Despite these problems 
and limitations, both the math and physics teachers report that t.he 
pre-requisite skills and concepts columns were a help in instruc- 
tional planning. The physics teacher also found the component 
tasks and environments columns helpful. The quality of the data 
in these columns, however, was criticized especially by the math 
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teacher who found, for example, that he often did not agree with the 
level of mathematics prescribed by the four column writer for the 
electronics skill given in the objective. He gave the following- 
example: "I may have felt that scientific notation was necessary, 
but the BO may only have addition and subtraction as a prerequisite. 
I felt this was very restrictive from /n- point of view." Obviously, 
this is a developric.-t problem, perfectly resolvable, and the kind of 
data development f.ijdback which the Project needs to systematically 
collect . 

When we look at the data from the point of view of the elcc-- 
tronics teacher, however, we g'^.^t a different picture. We must keep 
in mind that the data received by the pilot schools was "early data" 
that had not been fully developed, and had not been subjected to the 
later quality control procedures that purged the syscem of the worst 
of the initially developed BO's. Thus, what was pilot tested may 
be said to be of generally poor quality compared to what the Project 
was producing during the last six months. 

The electronics teacher was clearly enthusiastic about the 
data, found some imaginative ways to utilize it, and declared that 
the printouts had become his "lesson plans" and made his job 
easier. In the judgment of the cvaluators, the unhappy experience 
with the lap's may have stemmed from the poor quality of the LAP's 
and the lack of the Project's supporting technical assistance to 
provide for their improvement. But this failure proved to open 



the door for some serendipitous experimentation that revealed 
other possibilities for using the data in the classroom. We cite 
two examples briefly from this teacher's electronics class. 

A four column BO was copied on the blackboard. Students 
read the objective and accompanying four column information^ The 
objective was discussed to insure clarity of understanding. Students 
were then directed to complete the learning necessary to perform the 
task successfully. They could learn by any means they chcsc^ using 
the class and equipment as a laboratory and the teacher as a resource. 
When they were ready to be assessed on the objective, they requested 
an evaluation from the teacher. In this approach, the objective 
was shared with the students and they designed their own learning 
activities and requested assessment when ready. 

In another situation, the teacher provided the students with 
a radio needing repair. The students worked out the problem and 
the repairs, on their own, using the resources of the laboratory. 
After completing the task, they then were directed to the computer 
printouts of BO's and asked to search for performances that cor- 
responded to those they had completed in their work on the radio. 
These were incorporated into written student reports. Thus, the 
learning moved from experience back to the identification of tasks 
learned. Organizing the learning experience was accomplished through 
the availability of the objectives. 



NEESI's consultants feel that it is just such user experi- 
mentation that will eventually enable the Project to both improve 
the product and to develop user models that can be described and- 
packaged for other users. 

One final bit of reported feedback on the Springfield pilot 
is worth noting. The English teacher had some of the electronics 
students in one of her classes. She had developed some career 
exploration materials and activities, including some field trips 
to local industries. She found that, compared to the other students, 
the students from the electronics class made the best individual use 
of her role as a career information resource, and she felt that 
the linkage for this was due to her involvement as part of the 
teaching team using the electronics BO's. She was familiar with 
those BO's, and therefore could ro]. u to the students who were 
working with them in electronics class. We view this as evidence 
for the potential of the product in fostering interdisciplinary 
teaching that is occupat ionally relevant, 

Milford Pilot 

The Milford core teacher was a business teacher. He received 
a few hundred BO's covering the following occupations: legal secretary, 
secretary, medical secretax stenographer, file clerk, clerk general, 
clerk typist, bookkeeper, cashier, payroll clerk, and receptionist. 
On the team were teachers of physics, English, and social studies. 
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In addition to the schedu lin.-r problems which prevented the 
business students from being in the academic teachers' classes, there 
was an additional handicap in testing these data* The important 
concepts column from which academic teachers draw most of their 
teaching relevance for the occupations was blank on nearly all the 
bo's. In effect, there were no data for the academic teachers to 
voork with. As a result, they were forced to invent material that 
might support the work of the business teacher. 

The role of the physics teacher was the most difficult since 
the business students in the pilot test were only sophomores and 
could not take physics even if scheduling could have been arranged* 
In addition, as pointed out, there were no concepts with which any 
of the teachers could work. Each of the academic teachers, therefore, 
brainstorined their own ideas and developed some LAP*s that were 
business related, but not really connected to the BO*s. 

One experience, quite different for Milford than for Spring- 
field, was the success of the LAP's. PROJECT CAREER helped these 
teachers to "discover" LAP*s, and they continued to develop them for 
their own classes. The physics teacher was quite active in this 
approach and reported that his students were highly enthusiastic 
over the physics LAP's. This can be regarded as a positive benefit 
from the pilot experience, but not a test of the PROJECT CAP^EER 
product . 



rom 



As with Springfield, the only real product test came frc 
the occupational teacher. Again there was unequivocal enthusiasm 
for the BO'S. Compiling the responses from the quest xranaire completed 
by the business teacher in Milford. we can provide the following 
inf ormat ion: 

This teacher made fairly extensive uso of the BO's in his 
classroom. He estimated at least 50 hours of cias.rcom teaching 
time involved the PROJECT CAREER data. Ho developed several LAP's. 
each of which covered approximately seven BO's taken verbatim from 
the PROJECT CAREER printouts. He found the EC's to be precise 
enough to teach, and the performances relevant to in-school 
learning. The BO's were not organized and he would have preferred 
then in a sequence. 

His selection o:. the BO's to teach was based on his own 
curriculum as wei] vb.t he felt was relevant to his students. 
He felt the BO's were ••son.o help" in designing his instructional 
units. The ^ro^rogu is ite s;:m^_coUrmn he found to be "a guide 
in fulfilling the behavioral objective." The coj^ ponent task column 
"broke down the task into helpful sequential tasks." The alternative 
environments co Uum provided information on "the necessary tools 

to perform the tasks." There were not enough complete concepts 

columns to comment.. 

In general, this teacher commented that, "The BO's I used 
worked hand in hand with the textbook. I found the BO's very easy 



to work with and also the students I feel enjoyed them much more 
than the routine te^xtbook version.'* His recommendations for improving 
the product package included additional easily accessible support 
materials, and, for a full understanding of the program, an in-service 
training program for faculty members. 

Here is this teacher's entire sunamary comment: "I feel that 
my experience with the BO's that PROJECT CAREER issued me were 
excellent and a dynamite expedience not only for myself but from the 
students as v/ell. I would, however, admit I did not receive enough 
of them from PROJECT CAREER possibly due to the lack of communication 
at times. in my particular component I would have to say that 
the business area being the core, seemed to work out the best. A 
posciblc reason for this could have been the structured tine slots 
that the students v/erc placed in, not allowing them the freedom 
needed to see any of the four teachers that i:hey desired to sec on 
any given day. The business arpect of the 'team* worked out the 
best because the business instructor was present at all times to 
discuss and work with the LAP's with the students. In general all 
four disciplines v;orkcd out relatively well and I would say that 
the program v;as a successful procjram." 

Throughout our field interviews and surveys of all LEA 
pilot components connected ^;th PROJECT CAREER, the Third Party 
Evaluators have found this entliusiasm for the PROJECT CAREER product. 
We believe this is a remarkable and encouraging finding since the 



data were late in coming, not complete, of generally pooi qi -^liw , 
and the direction and support given to the LEA personnel - * 
sporadic, sometimes inconsistent, and not really adequate t .\ th^s 
tasks that were expected of these personnel. From that perspective, 
there could be no more clear demonstration of the potential value 
of this product for occupational education. 



seems to characterize the pilot testing of this Project comes fron 
the Milford teachers. Since the pilot model required the cooperative 
teamwork of the three academic teachers with the business: teacher, 
a number of discoveries were made. The business teacher /'discovered" 
the physics laboratory and developed a familiarity with it that, 
he said, enhanced his understanding of this part of the curriculum 
as well as that of his students. The academic teachers, forced to 
find (invent) wayj of developing subject matter units of relevance 
to the business students (even when they did not have them in 
class) found themselves stretched in interesting new ways. They 
developed a "team sense" in working together and understanding each 
other's inputs into the education of their students • They reported 
that this had been an invaluaVjle experience, ond they wanted to 
continue it. 



it substantiates the value of the aceidem.ic/occupat ional team, not 
central to the pilot test at this stage of the Project, but still 
s:.gnif icant . It is the judgment of the evaluators that a great 



A final example of the "success in failure'* experience that 



VJe find this kind of report to be salutary. For the Project, 
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a^^.j of credit for these results belongs to the LEA personnel 

who accomplishei them. The Project did, indeed, catalyze the 

experience, fund a summer workshop, and provide funds to the 

lea's to carry out t' e pilot, bMt it offered much less than was 

promised . 

SUMMARY OF 
FINDINGS 

The evalnator:. li^rlicve that the following summary statements 
rcprecont the most ealicnt aspects of our summative evaluative 
findings rclatii^ to the objectives of PROJECT CAREER/CAN. 

1. The PROJECT CAREER behavioral objective, with the 
additional four columns of curricular information added 
tc the throe part Magcr objective, represents a signifi- 
cant and innovative advance for occupational education. 
The four columns have demonstrable instructional value, 
and they arc readily transportable anywhere in the 
country . 

2. As of the tcrm.ination of this Project, the four colunjr. 
writing is not yet fully developed. Standards of 
quality control are still being developed. Additional 
manuals with important coding for full computer utili-- 
zation remain to bo dovolopcd. However, the production 
model, the training of writers, and the plans for 

dove loping the rema ini ivj manua Is arc complct o . 
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As of June 30, 1974, a data bank of four column 
objectives for the 116 occupations in the PROJECT 
CAREER list is about 9Cr/o complete. 

A sample data package for BO's in automobile mechanics 
indicates considerable variability in the extent and 
quality of information in the four columns. The two 
most variable columns were prc- requisite ski lis and 
compcnont tasks . This findinq is supported by the 
editors who report that these are the two most difficult 
columns to write. Completed manuals for these columns 
are also unavailable at thic time. 

Computer retrieval capability is now available for severa 
useful configurations of data. The lack of coded 
information for three of the four columns still limits 
retrieval capability at this time, but that is not a 
limitation of the computer. Most users will be able 
to request data in some useful format and receive it 
within two weeks • Some standard "on shelf" items will 
be maintained . 

A limited pilot tost of the product in two LEA class-- 
rooms, with supporting help from academic teachers, 
indicated that the data has high positive impact on 
users. This finding emerged despite obvious weaknesses 



in the data at were available for testing, as well as 
a host of other problems associated with the pilot 
testing • Feedback from the teachers indicates that 
the use of the data in classrooms also has good potential 
iippact on students ♦ The pilots, as conducted, did not 
provide for an adequate test of student impact. 
To summarize generally, the Third Party Evaluators find 
that the Project has established the feasibility of this product 
being useful for classroom instruction and curriculum development 
in occupational education and supporting academic areas. As the 
product was developed through June 30, 1974, it is not ready for full 
scale use and di^rsemination. In its current "raw" form, the product 
has boon improved in quality considerably over the Project's develop- 
menc life, but, as yet, there has been insufficient packaging 
development for users. Sequencing problems have not been dealt 
with, to cite a major remaining area of needed development. The 
four column information needs further refinement, quality control, 
and the complctioa . the remaining manuals which will contribute 
to tho aforementioned needs. 

With tho improvements and further developments sumimarized 
above, the Third Party Eviiluators feel that their findings would 
justify the conclusion that this BO product has the potential for 
high impact on users and student consumers of occupational education 
and that it will be fully transportable when further tested and 
developed . 
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VIII. RECnt.WP^^rrjr^ 

The rccorruncndations that will be listed here are bri.^f ^nd 
are inherent in the findings discussed above. 
NEESI recommends: 

1. That the Project not release any more data for testing 
until any batch to bo released has bc^n suD.-iected to 
additional editing and any gaps in infornauion 
have been filled. 

2. That the Project give high priority to completing the 
final rer.aining development steps for the four columns, 
na>T.ely the completion of the remaining nr.nuals and 
coding . 

3. That upon the completion of the remaining four column 
manuals, the completed four column objectives be re-edited 
and coding added where missing. 

4. That additional pxlot testing be backed up with careful 
in-scrvicc training for the personnel conducting the 
test, that regular follow-up contact be maintained to 
monxtor progress, and that adequate supporr personnel 
be made available to consult on instructional design 
development using the Project data. 

5. That additional programs and configurations of data 

retrieval be Umited until the inxtial retrieval programs 
can be tested for their feasibility and potential user 
demand. 



^2 r> 



That the Project made a major investment in developing 
a few alternative ways of sequencing and packaging the 
data in order to maximize the value and usefulness to 
potential users. 



IX. COST EFFECTIVENESS 

Cost ef f ectivensss considerations may be encompassed by three 
basic questions: 

1* What proportion of the resources were actually used 
to accomplish the objectives? 

2* What were the overall finite costs in relation to the 
objectives accomplished (outputs) ? 

3* Were there alternative wave to accomplish the objectives 
that would have been more cost effective? 

In order for evaluators to provide meaningful answers to 

any of these questions, at least two kinds of information are 

necessary: 

• A prograiTi budget containing dollar amounts related to 
specific Pro j ect object ives • 

• Accurate and corr.plete fiscal information relating 
disbursements to objectives as v;ell as line items. 

Neither of these conditions was met by any component of PROJECT- 
CAREER, with the exception of PROJECT CAREER/CAN which had a program 
budget . 

There w^as an effort made during the last months of the CAREER/ 
GUIDANCE project to institute budgeting by objectives for expen- 
ditures in each of the pilot LEA's. These budgets and related 
objectives vere developed by the LEA*s and submitted to the Project 
Administrator for approval. Disbursements were m.ade based on the 
attainment of or contribution to the attain'*ent of the local objec- 
tives* However, this constituted only a local arrangement with a 



rather sm-ill proportion of the total funds expended by PROJECT 
CAREER. Moreover, the records necessary to monitor these disburse- 
ments were not summarized and would have required a "mini-audit" of 
requisition forms* Ii any case, the Third Party Evaluators want to 
commend the Project Administrator for this effort at instituting a 
more cost effective system. 

Another difficulty with deternininc cost effectiveness for 
any specific component of PROJECT CAREER is the obvious fact that 
resources were interdependent such that it was impossible to 
separate out the effect of funding from one source from the effect 
on the same objective from another source. Central staff personnel 
seldom had single assignmen that focused 10C??i of their tim.e on 
one objective for one Project component • Two exam.ples will illustrate 
this fact. 

The administrator of CAREER/HANTflCAPFED dGvcued half or 
more of his time (variable different stages of the Project) to 
activities that were related to the data developiuent objectives of 
CAREEIVi3EVELO?:-IZ:HT? , yet his total salary was funded by CAREER/ 
EAXDICAPPED^ The Curriculum Ac^inistratcr for PROJECT CAREER was 
paid from CAR£ER/'DE\^EL0?yJ:7^ funds, yet a major portion of his tim.e 
was devoted to adm.inistering an.d directing the pilot activities 
associated with CAREE.VGVIDANCE , and most of the last year of his 
work on data developm.ent was devoted to four column writinc which 
was a funded objective under CAREER/CAN. 
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These problems of allocating resources to specific Projects 
with particularized objectives become even more complicated when we 
consider resources going into supplies, equipment rental, adminis- 
trative costs, etc. AS we pointed out, neither the fiscal records 
nor the budgets enabled any meaningful distinctions to be made in 
allocating resources. 

Such data on expenditures as was available to the evaluators 
in summary form (apart from actual vouchers and requisitions) con- 
sisted of two categories: 

1. An auditor's report on expenditures far all components 
of PROJECT CAREER fro.-. July 1, 1S71 through March 19, 
1973 • 

2. Bi-veekly fiscal reports from the accountant for all 
components based on a fiscal 1974 budget. The latent 
of these reports, dated June IS, 1974 formed the basis 
ror rinal ngures on Project expenditures. 

we reported elsetvhere on the fiscal management difficulties 

that beset the Project during its first two years. The auditor's 

report, dated June 22, 1973, contains the following statement: 

-At the Project office, internal control of expenditures, proper 

classification of charges., cor-.unicat ion of financial data to 

Recional School business office, aad audit trail of financial 

transactions to the accounting records was inadeq^aate in rr:any 

respects. These deficiencies are noted below through recorr-en- 

dations for improvement of the accounting system." 

The improvements were i-plemented by late April, 1973, cut 
there was stil' - l^'~< -i- v,., , 
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and over $600,000 had already been spent. Thus, tracing the effect 
of money spent during the first two fiscal years of PROJECT CAREER 
became almost impossible. An illustration of this problem can 
be made for CAREER/GUIDANCE. 

According to the June 1973 auditor's report, $112,003 was 
expended from the CAREER/GUIDATJCE budget through March 19, 1973. 
However, when the new line item budget for fiscal 1974 was drawn up 
and bi-weekly reports begun, the CAREER/GUIDANCE budget was drav^^n 
on the basis of the total funding for this Project from its incep- 
tion in July, 1972. Expenditures from the months preceding fiscal 
1974 were reallocated, apparently, to take account of the fact that 
this budget had been tapped for funds needed by other components, 
but which had not been available to PROJECT CAREER when expenditures 
were due. Thus, the audit shows aLT.ost $66,000 expended for per- 
sonnel at the Central Staff level, but the final CAREER/GUIDAIwE 
budget accounts for only $34,000 to Central Staff personnel. 
Thus, any spending prior to the last fiscal year of this Project 
cannot be accurately accounted for in relation to procres--- on 
attaining the objectives of this Project. 

In the opinion of the Third Party E\^aluators, the funding 
agencies are as r.uch t^ blarre for this state of affairs as is the 
Project adr.iniscration. For sor.c reason, the f undine: agencies did 
not insist on a program, budget at the tirr.e the proposals were made 
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or as a condition for the release of funds. (The exception to this 
was PROJECT CAREER/CAN) . 

This leaves only the possibility of the most gross estimates 
of cost effectiveness. For this purpose we have combined information 
on expenditures for all coir.ponents of PROJECT CAREER into one 
chart. Figure 2. We have, in turn, lumped all objectives into two 
categories, 1) data development and 2) pilot activity. Vfe have 
further arbitrarily assigned all expen^diturcs for personnel, supplies, 
etc*, that were essentially made at the Central Staff level to the 
objective of data development. Under data development we include 
the completion of S-part objectives (condition, performance, extent, 
prerequisite skills, component tasks, alternative environments, 
concepts, handicapped code) for 116 occupations and all necessary 
computer time, prograruminc , and output. This arrangement also 
assumes that lOyi: of the time of sta^ff was going into data develop- 
m.ent, an assumption that is obviously not true* We then identified 
all the funds that were expended directly to or for LEA personnel 
and their travel expenses. This m.oney we assur.ed represented the 
contribution of resources to the pilot activity. This assurri"otion 
is also not true since a portion of supplies, eguipm.ent rental, 
administrative costs, etc. ca.n ccrLainly be assigned to piloting. 
In the absence of any other basis for sorting out these costs, 
however, have r,3de this gross allocation. 



73 




ERIC 



« 



•J ^ 

fc« < 1. 


*A 

o 
o 
•V 


r«4 
tn 

•-4 


O 

c^ 
o 


1^ 

<-t 
o 


<^ 

fM 
o* 


r» 

o 
o 


O 

o 


u 

BC r: » 
< < 


rsi 

iTi 

r* 

O 


o 


m 
O 
m 




3 




1 








fM 




i*» 


vn 






<M 




X Q «^ 




a 

•-4 




«r 




rst 
•>4 






- J* 9 - 

< < 

U = U i- 


a 


o 

*o 
•i^ 

o 

%D 

» 


— 


m 
»n 


— 


O 

( 


3 
2> 
O 


< < < 

U ~ 


O 


O 

o 

C^ 




f-l 

n 
m 




"«r 
o 


O 
m 




O 






O 










U ^ ^ ^ 
K r: ~ 1 
< ^ I!^ 










n 




























S *^ — . 

«< ~ il^ "c 




^ — 

m 

ID 










n 

■V 


^ ^ c - 




if. 










«n 




5 II 










TN 




















<r 










> ' 
1^ •>* 










:f 


1 


1 


1 


1 


t 

111? 


1.*' 




1 

rr 






1 


















1 


















— 


1 














—1 


1 


i: 


1 i 


< 
< 










1 


■S 


= 1 




:^ I 
















1 f 






■< 












< 


















s 








< 




































^ 
* 


i 
















e fir «: 


21 d 






o 


2 


•3 > 





ERIC 



As can be seen from the chart, the bulk of funding for data 

developir.ent came from PROJECT CAREER/DEVELOPMENT and PROJECT CAREER/ 

JA^. Major funding for piloting came from PROJECT CAREER/HANDICAPPED 

and PROJECT CAREER/GUIDANCE. Of the total $1,123,542 expended by 

the Project through June 18, 1974, nearly $923,000 went to data 

development and a little more than $200,000 went to piloting. 

COST EFFECT I\*E!;ESS 
QUESTIONS 

1 _ t:rDrortion nf PROJl'CT CAREER resources were used to 
accoroli h the ob-^cctivcs of all Project comiponents ? In studying 
all th^ J. ^rts on expended funds that \%ero available to us, we 
xust conclude that there is no evidence to suggest that any part of 
the resources ot t:. s Project vore spent for other than the attain*- 
nent. cf Project objecri^es. The f.-^.ilure t:: budget by objectives 
r.akes it ir,possibie tc- identify ..-here cr cc what extent this 
conclusion rr.ight not. be true. There is other evidence discussed 
belov, hov;ever, that indiccites not all the pilot test resources 
vent into tr.e specifi-: objective of testinr: the data. 

VvTiat were the overrzll fin :! .tc costs in relation rp Prefect 
objectives or Project out-ut ? i-vc can say iron our gross figures 
that it -ver seOj.CC: to develop a da^a bank that we estina::e 

to be about ^CT, cor-pletc- in che for~ originally prrpc-sed v^nen the 
Project was funded in July, IC-T:. . The original funding prcposalr 
in which "cihc develcpr.ont cf ^his d^cz, bar;k was rrooosed for 
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coropletion over the three year period was funded to the extent of 
$577, 875. Over $400,000 had been spent on date, developr.ient fron all 
funding sources by the end of fiscal 1973 (two years into the three 
year funding period) . At that tine the data bank inclav.ed .^.bout 
550 completed objectives. There were also several thousand objectives 
at various other stages of the data develop.T.ent system. Also at 
this tiir^e the Project was working with the explicit target of 50,000 
objectives in the data Lank. In the opinion of the evaluators this 
was never a realistic figure and should be essentially discounted 
in looking the acccnplishr.-ents of the Project and the issue 
of cost effectiveness. But, fron the point of view of the stated 
objectives of the Project, the figure was there and certainly created 
an enormous gap between production and tarcct. 

During the last twelve ntonths of the Project, about $150,000 
in new development noney was spcat fron: :^ROJi::Cn: CAREER 'CA2^. This 
was added about CCD of rer.zininc data cevelopr.ent in CAREER^' 

DEVELOPME^n and another S 10 0,0 00 ^:.r3wn from CAREE?./ rlA2>Z)ICA?PED and 
CAREER/GUIEa:;CE, so, by our gross figures, about another $450,000 
went into data development during '::he last fiscal vear of the 
Project. The results were the addition cf about 9,000 nore 
completed BO's fcr the data bank and an additional three to four 
thousand B-O's i.: the systc^n needed to ccn:plcte all 11-0 occupations. 

To 
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Was PROJECT CAREER cost effective in producing this data 
bank of completed behavioral objectives ? The answer is probably 
"yes and no." We can say that the data bank was completed as 
promised in three years and that it cost approx innately $923,000 of 
a total budcet fron all sources of $1,400,915. That is probably 
acceptable for a research and developr:;ent project breakirxg new 
curricular territory to develop a product that will be nationally 
transportable with high potential inpact on users and student 
consur.ers . 

On the her harid, the proposal to develop this product 
'*'as funded for . 77,B75 for a three year period <-^nd so.-ne of those 
resources had to be used to secure nearly $400,000 nore dollars in 
order to cor.plete the data. It did, in face, require tw^o years and 
?400,j:o to learn how to produce a product that w^ould have reasonable 
C7uality and be useful for instruct ior. . A siniilar arount of nonev 
($45 3, CDC) ^.nd one r.ore year, was then required to cor.plete the data 
ban:< for about all 116 occupations on the PROJECT CAP.EER list. 

For ar.y reasonably definitive answer to t.^e cost: effective- 
ness of this Project we rr.ust turn to our final r-uestion. ICere there 
b Gt t er w a v . > to a c cctt 1 i s h the o b j ect jv e s t h 3t v: ru Id h av e been more 
cost effective ? To this question, the evaluzircrs believe the answer 
is a qualified ves . In this conte>rt: -oc- rust look at the second of the 
two -ajor objectives, the pilot testino of the oroduct . 
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Our gross figures indicate that about $200,000 was spent 
over a two year period to pilot test the usefulness of the data. 
Were the data ever pilot tested? The answer to this question is 
essentially no. Classroom use of the data was limited to two 
occupational classrooms using incomplete data for two occupational 
clusters (electronics and business) . No data bank with any of the 
computer retrieval possibilities was ever available for piloting. 
Data was never available in the form and quantity needed to ef f t.^.^tively 
test its use in guidance. Coded data for handicapped populations 
was never available in the quantity and quality needed for an 
adequate test • 

The undenied accomplishr;.Gr.t5 of Vhe pilot prcaram.s in the 
three LEA's and the enthusiasm for the potential of the product 
(the feasibility of v/hich was cerrainiy tested) , cannot obscirro 
the fact that the data were never addquatel^' ready for the testing 
that v;as to take place. V,'e car. say, then, that most u?f the resources 
going into the cilct program.s v;cr^ not focused on th*=- essential 
objective of testing the usefulness of the product for guidance, for 
the secondary classroor, and for scecial needs poi^ulat ions . 

This croblem., moreover, v:as e>±endcd to the data developm.ent 
prccess. Pilots that hac no data had be kept coinc on som.e basis. 
'tiC ha:vc no way of knovring , hoA-ever, to what extent data cevelopm^ent 
resourc-cs -iverc vrasted by the poor tim.ing of the oilot croaram,s • 
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We do kaow that the funding sources that produced money for the 
pilot test also added resources available for data development. 
Were the exigencies of proposal writing and the requirements of 
the funding agencies such that to get more money for data develop- 
ment piloting had to be included? The answer to this question is 
probably yes . 

In closing we want to err.phasize that the start up costs of 
a proje^ct of this nagnitude are bound to be high. In our judgment, 
the costs of this Project were fully warranted in relation to the 
innovations developed. 3y concentrating these funds on one project, 
a process and a product have been, developed that can now be duplicated 
by local school systeT.s throughout the country at a snail fraction 
of these origir.al developr.ent costs. 



X> CONCLUSION 

PROJECT CAREER/CAN was funded July 1, 1973, and in June, IS ^4, 
was evaluated on its three najor objectives which were to complete 
the addition cf four columns of inforniation to the PROJECT CAREER 
behavioral objective, to conputerize these dafcai for effective user 
retrieval, and to pilot test the use of the product in the classroom. 
All of these objectives were inherited from ongoing coniponents of 
PROJECT CAREER that began twelve to eichteen nonths before rhis 
Project was funded. The funding from this Project was entirely 
directed toward the first two objectives, with the third objective 
allowed to continue on its own from previous funding. The evaluative 
findings substantiate that the first two objectives were effectively 
accomplished and the product at this point is about 9yA com;plete. 
A sufficient data ban}; has been established tc warrant further 
testing. The pilor test escablished the feasibility cf usimg this 
product in the classroom:, but was too lim.ited and ineffectively 
moni^-cred to prnvice m^ore than rudim:entary data for further product 
develccment . 
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PROJECT CAREER/CAN FOOTNOTES 



1. PROJECT CAREER in Motion Addendum 
June, 1973 



2. PROJECT CAREER Conprehensive Project Plar. 
SepterJDer 1, 1972, page 1 



3. Ibid, Pages 1-2 



4. Izid, Pace 2 




Allan Epstoin 
May 16, 1974 



PROJECT CAREER REPORT 
SPRINGFI ELD SECONDARY PILOT 



Beginning in September of 1973, the teachers involved in the 
PROJECT CAREER Program began to use the Learning Activity Packages 
that had been developed the previous summer. 

Tno bulk of the information provided on the data sheets was 
well c. . ..ized and relevant to the objectives of the program. 
wa. vith ch. method of imparting the information that we ran intc 
difficulty. 

For the £ir=t five months we used the learning activity 
method. There were various problems that arose specif icall; .rom 
this format. 

From the very beginning, there was a problem rotating the 
■ther three teachers into the electron.ics class. Therefore, the 
in aim of relating certain aspects of the four columns with elec- 
tronics became very difficult. The needed reinforcement of any 
relationships became ineffective. 

Perhaps the original plan to coordinate the classes would 
hav. boon more effective. However, this alternative method of 
having rotating teacher, on an irregular basis (in the electronics 
Class) along with the regular science, math, and english classes 
was viewed by the students as an unfair overloading of their 



o 

ma 



schedules . 
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From a teaching point of view, I found the L.A.P. method 
left little room for variation and I became bored with the routine. 
Close contact with the students became difficult as most material 
was self-taught. The close contact with the student was threatened. 

Lastly, the time involved in preparing the L.A.P.'s and the 
execution of the lesson by the students war too long for the amount 
of material covered. 

Upon discussion with the other group members, it was decided 
that althov^vjh the behavioral objectives were valuable, the methods 
suggested were proving unsuccessful with these students. The format 
that we adopted is what follows, 

A behavioral objective from the data given us was placed on 
the board and discussed. Classwork was taken from school workbooks 
and textbooks. That helped to develop the particular objective. 
If it became apparent that a student was having difficulty with a 
certain algari'^hym, the subject teacher was notified and the student 
was given help. In this way the science, math, and english teachers 
became resource teachers and they were able to work on an individual 
basis with the students. The students seemed to respond better to 
more teacher involvemen,. , rather than tho irr^pcrsonal method of self- 
teaching . 

Guest speakers and field trips v;ere used to reiterate the 
relationships mentioned in class. Some of the guest speakers were 
I.B.M., Digital, and Hamilton Standard. 




As a further step we began to use the textbooks only as a 
reference book along with instruction manuals and schematics. The 
class was broken into subgroups. In each subgroup I arbitrarily' 
assigned a foreman with whom the others learned to cooperate. 
These groups were then assigned projects which involved not one but 
many behavioral objectives. The projects vere formed by grouping 
various objectives from the data sheets to form one task. Some of 
the tasks included building a trrnsistor radio, building a tube 
radio, setting up a television antenna with a rotor, building an 
electronic calculator, or changing a color television picture tube. 
Each of these projects can be viewed by the student as being essen- 
tial to anyone going into the electronics field. At the same time, 
the instructor can feel that necessary objectives have been met* 
Let us take two examples. 

In building a tube or transistor radio, the following 
objectives from the data sheet can be applied: 004913, 004914, 
004915,004918, 004917, 004924, 005174, 003115, 003114, and 1983. 

In changing and ali-jnir^q the tube of a color television 
and checking the circuits, the following objectives could be 
applied: 003115, 001982 001981, 001973, 001970. 

Wht»n each task was completed, a technical report was written 
by each student in the giuup. All applicable behavioral objectives 
were listed and discusreu, and the outcome of the project v.ms 
delineated. Some of th.^se reports arc included. 
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The students became enthusiastic about learning while 
doing. They were tired of learning theory. Now they could see why 
theory was important, first hand. Often, after completing a project, 
the group would look through the scan sheets and pick out objectives 
that they had mastered. They had to know the theory to use the 
equipment and to put the pieces of a project together. This was 
far more realistic to them than learning theory to pass a test. 
Their interest in going back to the scan sheets to dissect what thoy 
had and to try and further understand the procc .s involved rein*- 
forced my belief that this ricthod was valuable. 

There is no doubt that some of the drawbacks I have mentioned 
reflect to some degree my own inadequacy. The procedure originally 
decided upon could l;ave been more effective if a Tiore experienced 
teacher had written the L.A.P.'s. Some of my L.A.P.'s were, indeed, 
boring . 

To come extent, my lack of experience reflected in my ability 
to organize the other tcacherc involved into on effective unit. 

My inexperience, along with other drawbacks, could have been 
minimi.::ed if there had been more direction from the PROJECT CAREER 
Program. Field trips which were tc be planned to afford us the 
opportunity to obf^erve other schools engaged in this program never 
materialized. 

Most of the time v;e were left on our own to make decisions 
and to criticize our own programs. We were not observed by anyone 
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who could give any constructive criticism to our methods, or n>-ovide 
another view of which we were not aware. 

I feel the program has invaluable material to work with. ' 
Having dealt with it all for a year, I am in a better position to 
arrange a program for next year. It wouJd be a shame to disband aow 
when we have gathered much of the missing ingredients, the e-:pcrience, 
and wisdom to use the very valuable material that has been compiled. 
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Subject Area 

SECOVJDARY TFACHER3 SURVEY 



based oL^%^!-rn;\\rori:it^%t.L"^:f I'T^^" ^° ^^^^^ 
curriculum aad instructio^ objectives for use ir 

Objective and there other cLS^^b'e ^''^ ^ ^^i-PUfied 

of this evaluation that win suffice! ^'^^ purposes 

to in.tru"ctiorond curricuLTT '''^ ^ ^"^-^ is of vclu. 
tested several rchoolc u'^n ^^^^^'^^^ ^o b^- 

Preliminary evaluation h^s ;^^! xnterd.sciplinary tcan.s of t.-acher. 
with this evaluation, that tho\,--n.iM. l-T; Tl^ ^^^^^^^ 
^:choolo for piloting was n.t on "^ot " f "^"^^^ 
tnvcs into consideration thl ' X '^^^^^Taate. This evaluation 

on .UU^:^4'll ^.^.Tt'^a^lT' '""""^'^ ""^^-^ o„t 

I-rojfct will to fur^icir^nl ,v-,t io.V , „ =ucccs.-i o£ th 
rr^r.kly .,r,d Laiy you J^Z q>:=:^tion= 

unit for 0-. .r nr,Jc V'-'u l i ''r''' ' "'"'^^^ ^"^^ curr.cu u-,n 

this year that you d-.v-.toU ^ o u"r d ! ? 

toachin, froM Pc d.vc.lor,oci rSo'n I'^P^n^J or directly 
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(over) 



I£ they v.-cr<' r:i!':dornly arci<;p*/cd, did you otto:::pt to sequence 
thc;r.? v;;iat hiru of syztci did you u::c^ for nocjuoncing th.cia, 
if at nil? 



Did the l:0*.i yo-j rc voivcd all hc,vc the ^our coli.nHir. of j nf or 
n.ut ion rjd Jcd.- 



vro;: .iaI yo^ 
t :.:m.:1\;. 



i :7 yo^! h.id :.:cl: ouw to dcr i f^a cor.o :.nL:trucj 
ot:c;:y.;.L ^c-a:;!. ::Ml]r: 1 h:.t you could 
j:jb v.'itii: j--L tlv. v:: y/o^ i: as you Vvcrc: able 



to 



II you rc'-oiv-d a ].v:Lrh oi; 1:0 ' s vhat Iv.d the i:our colui.nn 
added, j)l/-.:.'o r: .:; ; '^^i to t]r..rc (:u-rLionn i:ccavd:iarr yoiT uro 
a!'>d the v::tj::ula . L o: r ac:i of ti:o four rol;:i;;n;;, 

la yoi.r i a;: Lracu io::u 1 i^lannii^r; the j^r. >.-rrMn t;- ito rhilln 

help hindraai'jo of no u'-ofulriCGs (circle one) 

E>:plain liov/ you irade ur-^c of tl;e ixr:e-rceui 3 ite r,]:ill:. colup.a 
in yc jr p] ana ia:j . 
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(over )V 



4 



tL'ic "^V" ^^^.^'^■-^-'-I--ri.once of teaching one or rr.ore of 

.pcc.fxc ...provoreal. xa your xr.ctrict ion, if .ny, resulted 
iron :\ji'.-xr:q the Bij'c? 



V.nat- L::,oci£:.c r-corr.T. n-iut i enrj v.culd hove lor Pro-jcrct C.-rocr to 

ool v.oule. no. only .xr.:l it o: c.rcaL iolo, buL -.vouKl ];r. 
cnthus.actxc alout inc^-pora'. .rvj th.^o into you^- 

'-"^^ ^ ^^or to .o;:::::-r: v..itton 

^^-^ ,;-n to U:;r. th :.;:,tcr i^ 1 c , in- 



(r:--ify); .K\;:ticn::]. ::u--;;o- ;..,tcrj 



•I..;;) re v ir. xzm:: 
-••■•tjrjc ; t .■■nr .c) ; etc . 



(> . U'ou J 
v.ho 



: ycni 



■r 1,1 
ill' 



t h~ 



Lm : ; j .:■ f c r i ; • .-• i-. r i ; c t iov 



or 



the 



"uliv:,:; 



■I I -r e]. 



il ?-"f.jcc:Lj vc u;: a uoin 
■'^ , V iiot ir: your pur.j tiuin 
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Editors' Manual 
4-Column Development 



Prepared and Duveloped by : 
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Witli The Assist a nce of : 

Gary Tirone 
Bob Tyrrell 
Peter Pettencill 

June. 1974 



Project CAREER 
Vincent P. Lamo, Director 
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4'COLUMN EDITORS' MANUAL 



:.t is the responsibility of the editor to be sure that all 4-column sheets 
reflect proper guidelines before being submitted for computerization. 

Part 1 

Guidehnes relative to all 4-Columns . 

1. Check that the proper I.D. No. is on each sheet. 

2. Be sure that only one letter or character (including punctuation) 
is entered in each biock» except for the fraction i which may be 
placed entirely in one block. The editor must also check to see 
that one blank box is left between each word or number • 

Example: 



AjNjsl WjElRI \=\\2\ J I jl| ,i |o|r! | ij /j 4 



When more than one line is needed for a statement, the editor 
should check that each line after the first indented one space, 

Example: 





Nipii: 


nItI, 


O'NlE! 


s: P 


a!c( e 


JV.'jH i E 




U sl - 








i\ 


N 


OlTlHI El Ri ' 


Ljl 


NIEI 1 i 


i 


1 







The editor should be sure that all words which must be continued 
to another line are separated by a hyphen. 

The editor must determine.- that both proper grammar and punctuation 
are used by the writer in all 4~column9. 



6, The editor must check for spelling errors. 

7, The editor must be sure that any abbreviations used are either 
trade abbreviations or are uraversally accepted. (Writers are 
encouraged not to use abbreviations when at all possible.) 
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Content Problems 



Content problems are not the job of the editor to? solve. .However, 
when editing the 4"'columns, continuity or consistency problems do 
arise. For example, the pre-requisite learning column might have 
"ability to use a drill", ycl. no indication of using a drill is men- 
tioned in the component task and/or the performance. Changes 
should not be made without first consulting the original 4'-column 
writer . 



Part II 

Guidel ines Relative to Each Individual Column 

Prc-Pvequisite Learnings Column: 

1. The editor must make sure that each 4- column writer has put 
"ability to", "use of\ or "knowledge of" before: each statement. 

2. if a pre-requisite math skill is taken from the Math Skill Inventory , 
the editor must be sure that the proper code number has been 
inserted in the spaces provided on the computer grid sheet. 

3. The editor must check for prerequisites covering occupational, 
mathematical, scientific and communication skill compelencit s 

Component I'ask'i Column: 

1 . The editor must check to see that each component statement begins 
with an action verb and is written in a concise statemei t rather than 
a narrative form . 

2. Component tasks should be checked to see if the steps have been 
properly sequenced. 

Environ. ir nt Column: 

1. The editor must check to be si re that commercial names are not 
used. 



2. 



Each tool or piece of equipment should be listed individually on a 
separate line and checked to see that it is in accord with the condi' 
t:ori, performAnce, and component tasks. 



3. One li..c should be skipped to distinguish environment and alternate 
environ •v,-.nt sections. 



Gjncepts Column: 



The eauor should check Vj ensure that the proper code number is 
being used with the piven concept and that the code number is correctly 
inserted in the spaces provided on the 4-cohjmn computer grid sheet. 



Part III 

9Hiij^ljIl«::g_/'^ '" ■^ Column Kdit'.rir'.I F oil n w-Up 
Purpose: 

The purpose of the -l-eolumn editr.rial follow-,--, is tc maintain a tiplit 
quality control over work of the 4-column writer. By contacting 
each vyrit-r. whether it be by telephone or t' rough the " l-Column 
Lditorial a.mment Form", the mistakes are held to a minimum. This 
will ultin-.»tely make tlu- writers job easier, as well as increase the 
production rate of the Completed 4-Columns. 

Mechanical Hditiru] Proctrdure: 

When usmn the "^-Column FdUorial Comment Form", the editor wiil 
abide by the following procedures: 

1. Insert the writers name in the space provided on ;.ho sheet. 

2. Indioat.. th.. problem location by checking one or cr. .re of the boxes 
.ah»:icd: Form. Column I. Collumn II. Column III. or Column IV. 

3. Indicate the specific problems in the area labeled "Editors Comments" 
•i. Number ..ach indivuival probh m ,f more than one problem ex.sts . 

^. Place his/ner signature and date in the space provided. 

IK') 
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6. Copy the completed "4" Column Editorial Comment Form" so both 
the editor and writer have copies. 

One to One Foilow-Up with Technical Writers: 

After a problem in the writer's work has been discovered and properly 
recorded, the editor should do the following: 

1. Make en appointment to meet with the writer. 

2. Supply the writer with a copy of th.? 4-column editorial comment form. 

3. Drscuss the various problems v/ith the writer, so that he can fully 
implement your sjig:^estions m his future work. 

4. Fill in the date on which the writer has been contacted, 

5. Make any additionr.l no.es which are deemed necessary. 

6. File the original copy in the writer's folder for future use. 

Sample 4-Ci>lumn Products Attached: 

1. A completed 8'part Project CARRCR Performance Objective 
/ 2. Four completed grid sheets. 

3. A 4- Column Editorial C' rnmfnt Form. 
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PRE-REQUISITE LEARNINGS 



In order to keep the scope sr^-requislte learnings under control, the 
followiny scheme should -L^rierec? to by the developi <t teams: 

1. Limit the pre-requisile^ .t-grnin^fs to the areas of moth and/or science: 
communications (Rcading/gp.gJish) ; and the occupational specialty. 

2. The pre-requi-.ice learnings listed should be those which cumulatively 
relate the hiohost levci pre-requislte in a given iirea needed for the 
performance of the behavior. 

Example: 

Conditions: Given a radial arm saw and a piece of x 8" x stock 

Performance: Cut a compound miter of 25 dcg'-ees of angle and 15 
degrees of t>cvcl 

Extent: Within ten minutes 

Pre-Requisi!e Learnings 



1, Identification end measurement of angles 

2, Use of protractor 

3, Use of sliding "T" bevel 
^, U«>c of radia! arm k^w 

5, Ability to measure accurately ♦ l/S" 



10 ; 

A-24 



ERIC 



1 



COMPONENT TASKS 

Ccmponent tasks are ordrred according to performance sequence. Each column 
becomes an ordered ta^.k analysis of the listed behavior and as such should 
reflect the activities which are imperaMve for the successful completion of the 
fisted be^idvior 



For the sample objective Nstf^d above, (he fonow;pg rn^ represent the sequence 
of tasks fc bo done pur:,'jant to its completion: ' 

Comp o nent Tas k s 

1. Set 'T'^ bevel 

2. M.->rk -stock 

3. Scle2ct prciocr bl^ide 

4. Acfjust orm of saA' to 25 degrees 

5. /Vijus* rotcr roading to 15 degrees 
Position stock on 

7. Don safety glasses 

Tc^ke tnol nick ar';d ccnfinn accurac/ 



8 



Where po.^.ibte, cfus*or ;toms in componcni colLimn \n or-^xir that we 

mijht save t.me (tly^rr^ple if for the ?bove rist of tasks, #6 can bo assumed 
to be a t.<k wh.cfi rerv.u'^s .:?.Mng t -l; ^\ - ^5, thet; xvc need only put down 
*6 - Posiljcn stock on saw) . 

The '^Corr^ponont Tr^ks'^ columr; dev.>loped bv the team member representing 
the occupation which r:l:-rs to the listed behavior. It wou!d appear, at this 
point, thr; other Ic.im n.cM-J^ers '.vould have minin,cii input. 

If there should ex.ct. for a g,ven bchovior, more than one seauence for 
co.'P^.letin;j Lehav'or. sooucnce whtch most clcsely relates to current 
practice ^h.^j!d b- included. 
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ENVIR0?sM3fHT 



Tho cnv. ronrrent- column represents a listing of needed tools and media as <^el 
as alternative aevices and sefJrgs which may be more conducive in enhancing 
the learning of 3 particular stjde.it. As such, the column should have the inpu 
of all men-,bcrs of the dcveioprr^cnt team, but cspccally the occupational instructc 
wno knosvs the required rraterials, etc., and a rredia specialist who can assist I 
team m the development of alternative environments. 

The "environment" listed for tnc ^bove examples is: 
Environment 

1 . Stock 

2. Protractor 

3. "T" bevel 

'i. Radial arm sew 

5. Scfcty giasser. 

6. Pencil 

7. Cross cut blade 

Aj^iin^ativ: -nviro-.r^ent. might 'snf! added to the list, .uch 35 cooperative 
education.; rp-cific programme.:! loxu certain multi-media presentations- 
simulation; etc. Therefc,-?, -.ve r: ght add: 

8. Use of assembly line environment 

9. Filrti en mo^s production 
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CONCEPTS 



The concepts- column ropreser.ts the efforts of the develooment tejm to provide 
those abstract rules, laws. generali7c,tions and principles which are related to 
behavior They w.ll also ^^cilU.te the transfer of learning and an mterdisciplina. 
approach to mMruct.on. It is d.s,c,ned to t.ke a .tudent from the concrete to the 
abstract relative to a given behavior. 

The column wi!, represent the inputs of the science and/cr m3Ch t.am ne.Tibers 
working closely with tl^c oc cupclional specialif^i. 

In developing tr.e cr.cepts. the .oam should include O'Wy the., concepts v^hich 
are directly reiotro to the- iisled^ bni^yior. 

A. While .^e ar. primarily involved with concepts in the sr.as of math and/or 
scene", you m,iy. vvncre appropriate, rndudc ccncppfs relating to law 
econc.tiics, etc. 

For the performance cK.Mtctive t-ing used as a-^ rxnmple, the concepts 
column v\ould contoin rhc fcHovvi-g: 

Concepts 

1» F^^ictforr 

2. ThermaJ Conductivity 

For a bchcvior in office- o.c.;p.Mions dealing wMh the lyp.na of a w;:i 
the concepts cclun-.n inaHt inciude: 

1. Probate 

2. Wilt and Tcstamcn': 

B. Concepts shoulo meet onp or more of the following definitions. 

Law. A rule derived Vom or a generalisation of observed phenomena 
and expr^nrnentel facts (Law or Conserv.-,tion of Energy) 

Princi£>:e. A scientific thoo^v. fact, or law of wice appUcation 
(Archlrr-eJes Principle) 

Cer,cn2li.aticn: The result n' ..neralizing: a notion, rule or law derived 
by analysis of ;nd.v.dual examples or instances, a aeneral infCncc 
an Induction. (Cas Pressure) or fClass.fic3f:cn) '"'^'"ence. 
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Rui e. A sUt^ent which belongs to the ordincry course of events or 
ccnclJtiun of a t*~.ing. That vvhich moy te expected, in the 
rvBjority of instance*. (Browninr^ .Votfor^) or fE<;'iiIihra;n) 

ld<?a : Any object cf ♦he n^ind oxistirg in thought, a ccncf^pl. notion or 
mental inprcsi:icn, 3 forr.rjJat'^o* thought. (Colo/ Fiftrahon) or 
(Spe-^d) 

If these cie'inittuns cio not apply to your copceptuj! stc:te7icnt, your slalemont 
dors not brfcng on your hst. If >/aur st^tennonf does mcot ^ny of Che above 
requ»ren;<:nts , ttst it by oniwern n f he foliowinr, q jcstioris: 

(1) Is yoL.r ::;t:^toment of o concept cJ?'-»r er^ouqh U^yr Instructrrs in different 
(instrjctlonct/tcchnicaP guMoc? cruris to uncV'-stand^ Would your 
statenv/rM b^: misundrr^^trnxi misinterprcrtcc:^ Is the idtvi ciear"* 
Rev.ritr ^itatcrnent if fv^cecr^ary. 

As an ryainple, a state^rrn^ -uro as - ^'Prr^.^o fluid helps transmit motion 
of brake podal to cr^kc stmo" - h.^s n'^^r.ing cutcmoJf\p specialists. 
If it were expressed as "norj-c?m;iresstbiIily of fluids" ii nas conceptual 
meanjf-^g in other ir'>v uLt»on.M ar.*es. 

(2) A*^e the'^c aoy other related ide.is or concepts that mean ths same thing 
in l^»is ca';>f? If you find a Lo^^tT 5tr.t»?Ticnt . ur.e tt. 

Cxamnle: We notice tf>at in cjtting our hoard, heat is generated. 

M»-?ny st:trnuntr c.jn de:^cribc thr,: heat, heat energy, 
friction, hp>( tran-.fer, h;t lay/ of thernodynamlci; , conser- 
\'ction cf en?r:)y Cf the<.e. liK- word ''friction'' is more 
•ipiiCific jn..^ Pill v<r-u!d be uvc^. 

13) CoulJ you c':^ndcnso yojr *if5tr.ncrt into one or two wc-ds? If so, do so. 

lixar.j^ti: rriction.' ' h' l:.-»:vn^?5 r»':cl'on and 

Co.if:wctiGn cf I^cot energy tcccn^rs T I rrra I Conduc it i ■ 

Refer to .-^ Ivcionce/Wath dictioorjry. , ^Jcec your concept neari what you 
think it r-.oaris? 

Example: Conductivity a\une :Tn,^n% other things (i.e. electricity), but 

by defining it c*^ thermal conc^jciti vity nny ambiguity is 
clirtiinated. 
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DATA FORMAT INSTRUCTIONS 



Write in the I.D, number of the behavioral objective with which you are working, 
This is the 6-dlgit number to the left of the behavioral objective. 
Example: ID-NO y 



0|02|7:9i2 i 



Indicate in the space assigned, that column which you are writing, using the 
follow ing codes: 

1 = Pre-requisite Learnings 

2 = Compcnent Skiiis 

3 = Environn-ie*-;! 
^ - Concepts 

Exair.p^e: (Jf you are v.orking with ''EnvirDOTtent") 
Column ! 3 I 



Write -only to the R:CHT OF TK£ DOL 3LE UNE. Leave spaces between words; 
s^tart each new itcra on a new 1?ne, and ii>dcnt by one space !f and when you 
need a second Iin3 for &n iter-n, 

PJease rr.^ke every effort to be legible, as t^e work you do will be read directly 
intc a comcuter. 



NOTE: On the ccripuler printout, mark to the right of the 
specify l>rne5v;or tne r.uirbcrjs} of column (s) com- 
pleted for the be^,av;cr5! cb^ccttve. 

Use no nor^ th^^i cr4e sheet r»5r cclur^n . 
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DEriMTION'S 



Pre-Requisite Learnings: 

Those skills, knowJerigss. undersiandmgs and attitude?; which are necessary 
for a stijd^-.t to in order to fac»litc;tc the acquisitson of the naw 

behavior and vsh.cn will not taught during the trer-trrent. 

CoTponrnt Tasks: 



The individual activities whkh c-e necessary for th^ cc-^piclion of the listed 
b€h2vior when tnat behavMor rnjst ha\e corrponent tasks oerformed in sequence, 

Ervircnrrent: 



The aJterrotive en\ ircnrrents which -nay be uti^iz^d for acquisiton c.^ the listed 
behavior, i.e. - fredia, rrodc of instruction. 



Conceci: 



The abstract r-!zs. prinripies. !^ws. and/or cener^iizctions which are related 
to t^^ p-rforrnarce cf th- ^;?iec: t-havior. These arc listed for purposes of 
prrvrdinr for the transfer ot ^earning and an inter-dls-cipiinary approach to 
instruction . 
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